Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1259 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4862. Min Act: -0.0000

Data Index: 1757486 (The Pile)

Max Activating Token Index: 940

Text #1

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4885. Min Act: -0.0000

Data Index: 1079178 (The Pile)

Max Activating Token Index: 257

Text #2

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4708. Min Act: -0.0000

Data Index: 1543180 (The Pile)

Max Activating Token Index: 492

Text #3

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4268. Min Act: -0.0000

Data Index: 1806799 (The Pile)

Max Activating Token Index: 1013

Text #4

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4038. Min Act: -0.0000

Data Index: 1959576 (The Pile)

Max Activating Token Index: 245

Text #5

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.4019. Min Act: -0.0000

Data Index: 181834 (The Pile)

Max Activating Token Index: 444

Text #6

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3753. Min Act: -0.0000

Data Index: 1560840 (The Pile)

Max Activating Token Index: 803

Text #7

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3941. Min Act: -0.0000

Data Index: 221763 (The Pile)

Max Activating Token Index: 980

Text #8

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3751. Min Act: -0.0000

Data Index: 631228 (The Pile)

Max Activating Token Index: 232

Text #9

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3603. Min Act: -0.0000

Data Index: 1643920 (The Pile)

Max Activating Token Index: 214

Text #10

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3550. Min Act: -0.0000

Data Index: 714981 (The Pile)

Max Activating Token Index: 810

Text #11

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3671. Min Act: -0.0000

Data Index: 625830 (The Pile)

Max Activating Token Index: 366

Text #12

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3169. Min Act: -0.0000

Data Index: 1857025 (The Pile)

Max Activating Token Index: 652

Text #13

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3491. Min Act: -0.0000

Data Index: 1192895 (The Pile)

Max Activating Token Index: 124

Text #14

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3459. Min Act: -0.0000

Data Index: 1265451 (The Pile)

Max Activating Token Index: 29

Text #15

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3339. Min Act: -0.0000

Data Index: 1966671 (The Pile)

Max Activating Token Index: 1005

Text #16

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3321. Min Act: -0.0000

Data Index: 1092002 (The Pile)

Max Activating Token Index: 135

Text #17

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3256. Min Act: -0.0000

Data Index: 1644816 (The Pile)

Max Activating Token Index: 811

Text #18

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3365. Min Act: -0.0000

Data Index: 903252 (The Pile)

Max Activating Token Index: 554

Text #19

Max Range: 0.4885. Min Range: -0.4885

Max Act: 0.3330. Min Act: -0.0000

Data Index: 1878699 (The Pile)

Max Activating Token Index: 975