Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1799 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.3284. Min Act: -0.0000

Data Index: 1421221 (The Pile)

Max Activating Token Index: 59

Text #1

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.1166. Min Act: -0.0000

Data Index: 219809 (The Pile)

Max Activating Token Index: 743

Text #2

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.2358. Min Act: -0.0000

Data Index: 195212 (The Pile)

Max Activating Token Index: 906

Text #3

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.1056. Min Act: -0.0000

Data Index: 419235 (The Pile)

Max Activating Token Index: 411

Text #4

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.0318. Min Act: -0.0000

Data Index: 204775 (The Pile)

Max Activating Token Index: 226

Text #5

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.0849. Min Act: -0.0000

Data Index: 6991 (The Pile)

Max Activating Token Index: 790

Text #6

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.9991. Min Act: -0.0000

Data Index: 1983247 (The Pile)

Max Activating Token Index: 9

Text #7

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.9892. Min Act: -0.0000

Data Index: 865366 (The Pile)

Max Activating Token Index: 516

Text #8

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.9389. Min Act: -0.0000

Data Index: 1840970 (The Pile)

Max Activating Token Index: 819

Text #9

Max Range: 1.3284. Min Range: -1.3284

Max Act: 1.0040. Min Act: -0.0000

Data Index: 989756 (The Pile)

Max Activating Token Index: 263

Text #10

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.9439. Min Act: -0.0000

Data Index: 1449437 (The Pile)

Max Activating Token Index: 409

Text #11

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.8669. Min Act: -0.0000

Data Index: 1858614 (The Pile)

Max Activating Token Index: 241

Text #12

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.8213. Min Act: -0.0000

Data Index: 1311933 (The Pile)

Max Activating Token Index: 253

Text #13

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.8215. Min Act: -0.0000

Data Index: 1485453 (The Pile)

Max Activating Token Index: 73

Text #14

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.7480. Min Act: -0.0000

Data Index: 841702 (The Pile)

Max Activating Token Index: 633

Text #15

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.8295. Min Act: -0.0000

Data Index: 128569 (The Pile)

Max Activating Token Index: 80

Text #16

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.7567. Min Act: -0.0000

Data Index: 518310 (The Pile)

Max Activating Token Index: 995

Text #17

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.7399. Min Act: -0.0000

Data Index: 1542533 (The Pile)

Max Activating Token Index: 808

Text #18

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.7573. Min Act: -0.0000

Data Index: 1953793 (The Pile)

Max Activating Token Index: 839

Text #19

Max Range: 1.3284. Min Range: -1.3284

Max Act: 0.7323. Min Act: -0.0000

Data Index: 1212843 (The Pile)

Max Activating Token Index: 123