Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3922 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.1004. Min Range: -3.1004

Max Act: 3.1004. Min Act: -0.0000

Data Index: 1057530 (The Pile)

Max Activating Token Index: 123

Text #1

Max Range: 3.1004. Min Range: -3.1004

Max Act: 3.0070. Min Act: -0.0000

Data Index: 771457 (The Pile)

Max Activating Token Index: 418

Text #2

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9879. Min Act: -0.0000

Data Index: 1358837 (The Pile)

Max Activating Token Index: 143

Text #3

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9985. Min Act: -0.0000

Data Index: 699917 (The Pile)

Max Activating Token Index: 432

Text #4

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8525. Min Act: -0.0000

Data Index: 337709 (The Pile)

Max Activating Token Index: 374

Text #5

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8207. Min Act: -0.0000

Data Index: 375479 (The Pile)

Max Activating Token Index: 702

Text #6

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9739. Min Act: -0.0000

Data Index: 130906 (The Pile)

Max Activating Token Index: 473

Text #7

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9414. Min Act: -0.0000

Data Index: 148269 (The Pile)

Max Activating Token Index: 825

Text #8

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9411. Min Act: -0.0000

Data Index: 546541 (The Pile)

Max Activating Token Index: 15

Text #9

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9198. Min Act: -0.0000

Data Index: 739433 (The Pile)

Max Activating Token Index: 252

Text #10

Max Range: 3.1004. Min Range: -3.1004

Max Act: 3.0664. Min Act: -0.0000

Data Index: 1769900 (The Pile)

Max Activating Token Index: 199

Text #11

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8860. Min Act: -0.0000

Data Index: 1675694 (The Pile)

Max Activating Token Index: 236

Text #12

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8324. Min Act: -0.0000

Data Index: 34537 (The Pile)

Max Activating Token Index: 730

Text #13

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8472. Min Act: -0.0000

Data Index: 1991285 (The Pile)

Max Activating Token Index: 658

Text #14

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8409. Min Act: -0.0000

Data Index: 1329402 (The Pile)

Max Activating Token Index: 636

Text #15

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8515. Min Act: -0.0000

Data Index: 1660906 (The Pile)

Max Activating Token Index: 977

Text #16

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.5970. Min Act: -0.0000

Data Index: 146639 (The Pile)

Max Activating Token Index: 466

Text #17

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8126. Min Act: -0.0000

Data Index: 141953 (The Pile)

Max Activating Token Index: 919

Text #18

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.8058. Min Act: -0.0000

Data Index: 1508832 (The Pile)

Max Activating Token Index: 401

Text #19

Max Range: 3.1004. Min Range: -3.1004

Max Act: 2.9456. Min Act: -0.0000

Data Index: 1923100 (The Pile)

Max Activating Token Index: 351