Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 865 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.8061. Min Act: -0.0000

Data Index: 919856 (The Pile)

Max Activating Token Index: 335

Text #1

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.8487. Min Act: -0.0000

Data Index: 140847 (The Pile)

Max Activating Token Index: 402

Text #2

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.7500. Min Act: -0.0000

Data Index: 877452 (The Pile)

Max Activating Token Index: 970

Text #3

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.7485. Min Act: -0.0000

Data Index: 1901424 (The Pile)

Max Activating Token Index: 389

Text #4

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6917. Min Act: -0.0000

Data Index: 1202583 (The Pile)

Max Activating Token Index: 940

Text #5

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6981. Min Act: -0.0000

Data Index: 904741 (The Pile)

Max Activating Token Index: 133

Text #6

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6476. Min Act: -0.0000

Data Index: 1028345 (The Pile)

Max Activating Token Index: 886

Text #7

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6858. Min Act: -0.0000

Data Index: 1479047 (The Pile)

Max Activating Token Index: 376

Text #8

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.7125. Min Act: -0.0000

Data Index: 884531 (The Pile)

Max Activating Token Index: 195

Text #9

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6783. Min Act: -0.0000

Data Index: 1819690 (The Pile)

Max Activating Token Index: 447

Text #10

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6207. Min Act: -0.0000

Data Index: 944421 (The Pile)

Max Activating Token Index: 213

Text #11

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6315. Min Act: -0.0000

Data Index: 946621 (The Pile)

Max Activating Token Index: 365

Text #12

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6286. Min Act: -0.0000

Data Index: 547307 (The Pile)

Max Activating Token Index: 1020

Text #13

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6236. Min Act: -0.0000

Data Index: 107053 (The Pile)

Max Activating Token Index: 589

Text #14

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6022. Min Act: -0.0000

Data Index: 1228498 (The Pile)

Max Activating Token Index: 137

Text #15

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6621. Min Act: -0.0000

Data Index: 545927 (The Pile)

Max Activating Token Index: 690

Text #16

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6208. Min Act: -0.0000

Data Index: 1170152 (The Pile)

Max Activating Token Index: 382

Text #17

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.5958. Min Act: -0.0000

Data Index: 551627 (The Pile)

Max Activating Token Index: 984

Text #18

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.5931. Min Act: -0.0000

Data Index: 1366897 (The Pile)

Max Activating Token Index: 31

Text #19

Max Range: 0.8487. Min Range: -0.8487

Max Act: 0.6465. Min Act: -0.0000

Data Index: 1301077 (The Pile)

Max Activating Token Index: 514