Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2935 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.8433. Min Act: -0.0000

Data Index: 321127 (The Pile)

Max Activating Token Index: 629

Text #1

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5897. Min Act: -0.0000

Data Index: 1673989 (The Pile)

Max Activating Token Index: 378

Text #2

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.6453. Min Act: 0.0000

Data Index: 1078714 (The Pile)

Max Activating Token Index: 44

Text #3

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5831. Min Act: -0.0000

Data Index: 685624 (The Pile)

Max Activating Token Index: 307

Text #4

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5070. Min Act: -0.0000

Data Index: 1392869 (The Pile)

Max Activating Token Index: 96

Text #5

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5349. Min Act: -0.0000

Data Index: 1324646 (The Pile)

Max Activating Token Index: 185

Text #6

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5652. Min Act: -0.0000

Data Index: 1918538 (The Pile)

Max Activating Token Index: 907

Text #7

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5570. Min Act: -0.0000

Data Index: 1956367 (The Pile)

Max Activating Token Index: 565

Text #8

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5382. Min Act: -0.0000

Data Index: 1530935 (The Pile)

Max Activating Token Index: 672

Text #9

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4679. Min Act: -0.0000

Data Index: 734309 (The Pile)

Max Activating Token Index: 494

Text #10

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.5105. Min Act: -0.0000

Data Index: 1835285 (The Pile)

Max Activating Token Index: 87

Text #11

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4672. Min Act: -0.0000

Data Index: 576153 (The Pile)

Max Activating Token Index: 472

Text #12

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4698. Min Act: -0.0000

Data Index: 1211724 (The Pile)

Max Activating Token Index: 1005

Text #13

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4611. Min Act: -0.0000

Data Index: 1567370 (The Pile)

Max Activating Token Index: 1016

Text #14

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4935. Min Act: -0.0000

Data Index: 1242105 (The Pile)

Max Activating Token Index: 511

Text #15

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4462. Min Act: -0.0000

Data Index: 1845257 (The Pile)

Max Activating Token Index: 19

Text #16

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4758. Min Act: -0.0000

Data Index: 893008 (The Pile)

Max Activating Token Index: 47

Text #17

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4571. Min Act: -0.0000

Data Index: 1069109 (The Pile)

Max Activating Token Index: 861

Text #18

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4771. Min Act: -0.0000

Data Index: 762006 (The Pile)

Max Activating Token Index: 38

Text #19

Max Range: 0.8433. Min Range: -0.8433

Max Act: 0.4523. Min Act: -0.0000

Data Index: 825305 (The Pile)

Max Activating Token Index: 47