Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3308 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.8359. Min Act: -0.0000

Data Index: 994578 (The Pile)

Max Activating Token Index: 518

Text #1

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.8252. Min Act: -0.0000

Data Index: 1797303 (The Pile)

Max Activating Token Index: 426

Text #2

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.8497. Min Act: -0.0000

Data Index: 1894140 (The Pile)

Max Activating Token Index: 896

Text #3

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.8274. Min Act: -0.0000

Data Index: 426068 (The Pile)

Max Activating Token Index: 777

Text #4

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.7629. Min Act: -0.0000

Data Index: 1082519 (The Pile)

Max Activating Token Index: 830

Text #5

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.7463. Min Act: -0.0000

Data Index: 769456 (The Pile)

Max Activating Token Index: 990

Text #6

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.7629. Min Act: -0.0000

Data Index: 543963 (The Pile)

Max Activating Token Index: 232

Text #7

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.7064. Min Act: -0.0000

Data Index: 812885 (The Pile)

Max Activating Token Index: 95

Text #8

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.6787. Min Act: -0.0000

Data Index: 1082829 (The Pile)

Max Activating Token Index: 132

Text #9

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.6780. Min Act: -0.0000

Data Index: 90739 (The Pile)

Max Activating Token Index: 284

Text #10

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.7141. Min Act: -0.0000

Data Index: 1973929 (The Pile)

Max Activating Token Index: 366

Text #11

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.6413. Min Act: -0.0000

Data Index: 374341 (The Pile)

Max Activating Token Index: 721

Text #12

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.6439. Min Act: -0.0000

Data Index: 199500 (The Pile)

Max Activating Token Index: 982

Text #13

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5780. Min Act: -0.0000

Data Index: 1949055 (The Pile)

Max Activating Token Index: 815

Text #14

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5958. Min Act: -0.0000

Data Index: 1018843 (The Pile)

Max Activating Token Index: 616

Text #15

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5922. Min Act: -0.0000

Data Index: 747164 (The Pile)

Max Activating Token Index: 978

Text #16

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5431. Min Act: -0.0000

Data Index: 1742008 (The Pile)

Max Activating Token Index: 770

Text #17

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5546. Min Act: -0.0000

Data Index: 1716258 (The Pile)

Max Activating Token Index: 773

Text #18

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5656. Min Act: -0.0000

Data Index: 1199776 (The Pile)

Max Activating Token Index: 499

Text #19

Max Range: 0.8497. Min Range: -0.8497

Max Act: 0.5357. Min Act: -0.0000

Data Index: 1668591 (The Pile)

Max Activating Token Index: 803