Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 461 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.9015. Min Act: -0.0000

Data Index: 1015028 (The Pile)

Max Activating Token Index: 901

Text #1

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.7243. Min Act: -0.0000

Data Index: 1387943 (The Pile)

Max Activating Token Index: 958

Text #2

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6746. Min Act: -0.0000

Data Index: 110638 (The Pile)

Max Activating Token Index: 986

Text #3

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6715. Min Act: -0.0000

Data Index: 1620323 (The Pile)

Max Activating Token Index: 333

Text #4

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6495. Min Act: -0.0000

Data Index: 1821877 (The Pile)

Max Activating Token Index: 655

Text #5

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.7166. Min Act: -0.0000

Data Index: 161701 (The Pile)

Max Activating Token Index: 724

Text #6

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6229. Min Act: -0.0000

Data Index: 628188 (The Pile)

Max Activating Token Index: 87

Text #7

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.5980. Min Act: -0.0000

Data Index: 1390833 (The Pile)

Max Activating Token Index: 742

Text #8

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6758. Min Act: -0.0000

Data Index: 484353 (The Pile)

Max Activating Token Index: 452

Text #9

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6573. Min Act: -0.0000

Data Index: 159738 (The Pile)

Max Activating Token Index: 197

Text #10

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6037. Min Act: -0.0000

Data Index: 1987690 (The Pile)

Max Activating Token Index: 842

Text #11

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6361. Min Act: -0.0000

Data Index: 7208 (The Pile)

Max Activating Token Index: 282

Text #12

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.5990. Min Act: -0.0000

Data Index: 1642115 (The Pile)

Max Activating Token Index: 724

Text #13

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6059. Min Act: -0.0000

Data Index: 983325 (The Pile)

Max Activating Token Index: 67

Text #14

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6384. Min Act: -0.0000

Data Index: 794106 (The Pile)

Max Activating Token Index: 483

Text #15

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6574. Min Act: -0.0000

Data Index: 1020520 (The Pile)

Max Activating Token Index: 89

Text #16

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6689. Min Act: -0.0000

Data Index: 284139 (The Pile)

Max Activating Token Index: 592

Text #17

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6148. Min Act: -0.0000

Data Index: 1182701 (The Pile)

Max Activating Token Index: 367

Text #18

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.5882. Min Act: -0.0000

Data Index: 1262163 (The Pile)

Max Activating Token Index: 231

Text #19

Max Range: 0.9015. Min Range: -0.9015

Max Act: 0.6235. Min Act: -0.0000

Data Index: 1468764 (The Pile)

Max Activating Token Index: 403