Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3966 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.7166. Min Act: -0.0000

Data Index: 13060 (The Pile)

Max Activating Token Index: 929

Text #1

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.6358. Min Act: -0.0000

Data Index: 566187 (The Pile)

Max Activating Token Index: 464

Text #2

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.6355. Min Act: -0.0000

Data Index: 1971017 (The Pile)

Max Activating Token Index: 241

Text #3

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5782. Min Act: -0.0000

Data Index: 1582033 (The Pile)

Max Activating Token Index: 149

Text #4

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5335. Min Act: -0.0000

Data Index: 1557162 (The Pile)

Max Activating Token Index: 172

Text #5

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5354. Min Act: -0.0000

Data Index: 1664538 (The Pile)

Max Activating Token Index: 869

Text #6

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5304. Min Act: -0.0000

Data Index: 687418 (The Pile)

Max Activating Token Index: 863

Text #7

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5347. Min Act: -0.0000

Data Index: 875590 (The Pile)

Max Activating Token Index: 743

Text #8

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5204. Min Act: -0.0000

Data Index: 1477297 (The Pile)

Max Activating Token Index: 255

Text #9

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4969. Min Act: -0.0000

Data Index: 507490 (The Pile)

Max Activating Token Index: 574

Text #10

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5355. Min Act: -0.0000

Data Index: 611507 (The Pile)

Max Activating Token Index: 293

Text #11

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5005. Min Act: -0.0000

Data Index: 1829372 (The Pile)

Max Activating Token Index: 143

Text #12

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5002. Min Act: -0.0000

Data Index: 1464605 (The Pile)

Max Activating Token Index: 374

Text #13

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4854. Min Act: -0.0000

Data Index: 442180 (The Pile)

Max Activating Token Index: 1002

Text #14

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4950. Min Act: -0.0000

Data Index: 641083 (The Pile)

Max Activating Token Index: 490

Text #15

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4932. Min Act: -0.0000

Data Index: 1999410 (The Pile)

Max Activating Token Index: 470

Text #16

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.5067. Min Act: -0.0000

Data Index: 1487364 (The Pile)

Max Activating Token Index: 298

Text #17

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4576. Min Act: -0.0000

Data Index: 1689299 (The Pile)

Max Activating Token Index: 243

Text #18

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4781. Min Act: -0.0000

Data Index: 1335334 (The Pile)

Max Activating Token Index: 264

Text #19

Max Range: 0.7166. Min Range: -0.7166

Max Act: 0.4603. Min Act: -0.0000

Data Index: 867300 (The Pile)

Max Activating Token Index: 329