Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 854 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.1344. Min Range: -1.1344

Max Act: 1.1344. Min Act: -0.0000

Data Index: 16414 (The Pile)

Max Activating Token Index: 48

Text #1

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9390. Min Act: -0.0000

Data Index: 219336 (The Pile)

Max Activating Token Index: 868

Text #2

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.9165. Min Act: -0.0000

Data Index: 1178866 (The Pile)

Max Activating Token Index: 253

Text #3

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8748. Min Act: -0.0000

Data Index: 924487 (The Pile)

Max Activating Token Index: 190

Text #4

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8349. Min Act: -0.0000

Data Index: 1827102 (The Pile)

Max Activating Token Index: 696

Text #5

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7901. Min Act: -0.0000

Data Index: 1788112 (The Pile)

Max Activating Token Index: 133

Text #6

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8572. Min Act: -0.0000

Data Index: 821087 (The Pile)

Max Activating Token Index: 743

Text #7

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7680. Min Act: -0.0000

Data Index: 1702496 (The Pile)

Max Activating Token Index: 198

Text #8

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8279. Min Act: -0.0000

Data Index: 1866894 (The Pile)

Max Activating Token Index: 302

Text #9

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8084. Min Act: -0.0000

Data Index: 1282308 (The Pile)

Max Activating Token Index: 519

Text #10

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7011. Min Act: -0.0000

Data Index: 1704267 (The Pile)

Max Activating Token Index: 421

Text #11

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.8099. Min Act: -0.0000

Data Index: 1020270 (The Pile)

Max Activating Token Index: 147

Text #12

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7396. Min Act: -0.0000

Data Index: 40312 (The Pile)

Max Activating Token Index: 95

Text #13

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7227. Min Act: -0.0000

Data Index: 1285423 (The Pile)

Max Activating Token Index: 274

Text #14

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7852. Min Act: -0.0000

Data Index: 237784 (The Pile)

Max Activating Token Index: 296

Text #15

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7093. Min Act: -0.0000

Data Index: 1842064 (The Pile)

Max Activating Token Index: 722

Text #16

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7322. Min Act: -0.0000

Data Index: 922833 (The Pile)

Max Activating Token Index: 41

Text #17

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.6243. Min Act: -0.0000

Data Index: 1971731 (The Pile)

Max Activating Token Index: 993

Text #18

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.6890. Min Act: -0.0000

Data Index: 1959254 (The Pile)

Max Activating Token Index: 206

Text #19

Max Range: 1.1344. Min Range: -1.1344

Max Act: 0.7482. Min Act: -0.0000

Data Index: 839172 (The Pile)

Max Activating Token Index: 165