Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1249 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.9505. Min Act: -0.0000

Data Index: 1488117 (The Pile)

Max Activating Token Index: 271

Text #1

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.9206. Min Act: -0.0000

Data Index: 1170922 (The Pile)

Max Activating Token Index: 606

Text #2

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7459. Min Act: -0.0000

Data Index: 1606314 (The Pile)

Max Activating Token Index: 158

Text #3

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7869. Min Act: -0.0000

Data Index: 896050 (The Pile)

Max Activating Token Index: 192

Text #4

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7340. Min Act: -0.0000

Data Index: 1810262 (The Pile)

Max Activating Token Index: 228

Text #5

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.8524. Min Act: -0.0000

Data Index: 654566 (The Pile)

Max Activating Token Index: 488

Text #6

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.8890. Min Act: -0.0000

Data Index: 1817797 (The Pile)

Max Activating Token Index: 177

Text #7

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.8152. Min Act: -0.0000

Data Index: 1938937 (The Pile)

Max Activating Token Index: 408

Text #8

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.8247. Min Act: -0.0000

Data Index: 1408582 (The Pile)

Max Activating Token Index: 281

Text #9

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7428. Min Act: -0.0000

Data Index: 297929 (The Pile)

Max Activating Token Index: 168

Text #10

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.6802. Min Act: -0.0000

Data Index: 1885291 (The Pile)

Max Activating Token Index: 151

Text #11

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7129. Min Act: -0.0000

Data Index: 1613970 (The Pile)

Max Activating Token Index: 314

Text #12

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7281. Min Act: -0.0000

Data Index: 1159277 (The Pile)

Max Activating Token Index: 751

Text #13

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7409. Min Act: -0.0000

Data Index: 1869927 (The Pile)

Max Activating Token Index: 512

Text #14

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.6444. Min Act: -0.0000

Data Index: 1906427 (The Pile)

Max Activating Token Index: 184

Text #15

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.5817. Min Act: -0.0000

Data Index: 1731529 (The Pile)

Max Activating Token Index: 1013

Text #16

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.7223. Min Act: -0.0000

Data Index: 1797188 (The Pile)

Max Activating Token Index: 616

Text #17

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.6736. Min Act: -0.0000

Data Index: 1076145 (The Pile)

Max Activating Token Index: 38

Text #18

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.6524. Min Act: -0.0000

Data Index: 1920058 (The Pile)

Max Activating Token Index: 992

Text #19

Max Range: 0.9505. Min Range: -0.9505

Max Act: 0.6406. Min Act: -0.0000

Data Index: 312864 (The Pile)

Max Activating Token Index: 234