Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1966 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.9673. Min Act: -0.0000

Data Index: 1892697 (The Pile)

Max Activating Token Index: 814

Text #1

Max Range: 1.0035. Min Range: -1.0035

Max Act: 1.0035. Min Act: -0.0000

Data Index: 1420701 (The Pile)

Max Activating Token Index: 291

Text #2

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.9763. Min Act: -0.0000

Data Index: 1779914 (The Pile)

Max Activating Token Index: 144

Text #3

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.9558. Min Act: -0.0000

Data Index: 1420050 (The Pile)

Max Activating Token Index: 681

Text #4

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8996. Min Act: -0.0000

Data Index: 1201752 (The Pile)

Max Activating Token Index: 778

Text #5

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8879. Min Act: -0.0000

Data Index: 1524709 (The Pile)

Max Activating Token Index: 330

Text #6

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8808. Min Act: -0.0000

Data Index: 726816 (The Pile)

Max Activating Token Index: 370

Text #7

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8023. Min Act: -0.0000

Data Index: 1755687 (The Pile)

Max Activating Token Index: 525

Text #8

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.9040. Min Act: -0.0000

Data Index: 845002 (The Pile)

Max Activating Token Index: 244

Text #9

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8527. Min Act: -0.0000

Data Index: 1321563 (The Pile)

Max Activating Token Index: 848

Text #10

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8316. Min Act: -0.0000

Data Index: 1672379 (The Pile)

Max Activating Token Index: 383

Text #11

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.7671. Min Act: -0.0000

Data Index: 1553673 (The Pile)

Max Activating Token Index: 695

Text #12

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8166. Min Act: -0.0000

Data Index: 724493 (The Pile)

Max Activating Token Index: 966

Text #13

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8203. Min Act: -0.0000

Data Index: 1465438 (The Pile)

Max Activating Token Index: 354

Text #14

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8492. Min Act: -0.0000

Data Index: 401153 (The Pile)

Max Activating Token Index: 707

Text #15

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.7979. Min Act: -0.0000

Data Index: 749948 (The Pile)

Max Activating Token Index: 496

Text #16

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.7424. Min Act: -0.0000

Data Index: 20152 (The Pile)

Max Activating Token Index: 108

Text #17

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8356. Min Act: -0.0000

Data Index: 1763661 (The Pile)

Max Activating Token Index: 752

Text #18

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.8136. Min Act: -0.0000

Data Index: 1388576 (The Pile)

Max Activating Token Index: 193

Text #19

Max Range: 1.0035. Min Range: -1.0035

Max Act: 0.7516. Min Act: -0.0000

Data Index: 1430564 (The Pile)

Max Activating Token Index: 169