Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2830 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0653. Min Act: 0.0000

Data Index: 1033664 (The Pile)

Max Activating Token Index: 2

Text #1

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0504. Min Act: 0.0000

Data Index: 259614 (The Pile)

Max Activating Token Index: 2

Text #2

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0498. Min Act: 0.0000

Data Index: 656972 (The Pile)

Max Activating Token Index: 2

Text #3

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0493. Min Act: 0.0000

Data Index: 1378088 (The Pile)

Max Activating Token Index: 2

Text #4

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0451. Min Act: -0.0000

Data Index: 1620456 (The Pile)

Max Activating Token Index: 2

Text #5

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0443. Min Act: 0.0000

Data Index: 370009 (The Pile)

Max Activating Token Index: 2

Text #6

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0440. Min Act: 0.0000

Data Index: 632400 (The Pile)

Max Activating Token Index: 2

Text #7

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0437. Min Act: -0.0000

Data Index: 64246 (The Pile)

Max Activating Token Index: 2

Text #8

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0434. Min Act: 0.0000

Data Index: 496539 (The Pile)

Max Activating Token Index: 5

Text #9

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0430. Min Act: 0.0001

Data Index: 129328 (The Pile)

Max Activating Token Index: 3

Text #10

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0410. Min Act: -0.0000

Data Index: 961367 (The Pile)

Max Activating Token Index: 3

Text #11

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0410. Min Act: 0.0000

Data Index: 1853702 (The Pile)

Max Activating Token Index: 2

Text #12

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0409. Min Act: 0.0000

Data Index: 1255136 (The Pile)

Max Activating Token Index: 3

Text #13

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0416. Min Act: 0.0000

Data Index: 230407 (The Pile)

Max Activating Token Index: 2

Text #14

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0387. Min Act: 0.0000

Data Index: 1938102 (The Pile)

Max Activating Token Index: 2

Text #15

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0399. Min Act: 0.0000

Data Index: 982671 (The Pile)

Max Activating Token Index: 2

Text #16

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0399. Min Act: 0.0000

Data Index: 129130 (The Pile)

Max Activating Token Index: 2

Text #17

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0400. Min Act: -0.0000

Data Index: 881330 (The Pile)

Max Activating Token Index: 3

Text #18

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0398. Min Act: -0.0000

Data Index: 846231 (The Pile)

Max Activating Token Index: 2

Text #19

Max Range: 0.0653. Min Range: -0.0653

Max Act: 0.0396. Min Act: -0.0000

Data Index: 1947358 (The Pile)

Max Activating Token Index: 2