Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3585 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.7596. Min Act: -0.0000

Data Index: 123606 (The Pile)

Max Activating Token Index: 859

Text #1

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4497. Min Act: -0.0000

Data Index: 277160 (The Pile)

Max Activating Token Index: 933

Text #2

Max Range: 3.7596. Min Range: -3.7596

Max Act: 3.4353. Min Act: -0.0000

Data Index: 1217046 (The Pile)

Max Activating Token Index: 803

Text #3

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.7233. Min Act: -0.0000

Data Index: 1198567 (The Pile)

Max Activating Token Index: 1021

Text #4

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.6205. Min Act: -0.0000

Data Index: 52601 (The Pile)

Max Activating Token Index: 764

Text #5

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.6917. Min Act: -0.0000

Data Index: 1405122 (The Pile)

Max Activating Token Index: 1009

Text #6

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.8628. Min Act: -0.0000

Data Index: 297524 (The Pile)

Max Activating Token Index: 908

Text #7

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.7648. Min Act: -0.0000

Data Index: 1607408 (The Pile)

Max Activating Token Index: 280

Text #8

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.7058. Min Act: -0.0000

Data Index: 177196 (The Pile)

Max Activating Token Index: 636

Text #9

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.6248. Min Act: -0.0000

Data Index: 879476 (The Pile)

Max Activating Token Index: 886

Text #10

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.5769. Min Act: -0.0000

Data Index: 1255498 (The Pile)

Max Activating Token Index: 931

Text #11

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.4673. Min Act: -0.0000

Data Index: 328411 (The Pile)

Max Activating Token Index: 632

Text #12

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.2399. Min Act: -0.0000

Data Index: 258057 (The Pile)

Max Activating Token Index: 193

Text #13

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.1878. Min Act: -0.0000

Data Index: 108804 (The Pile)

Max Activating Token Index: 707

Text #14

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.3454. Min Act: -0.0000

Data Index: 1523435 (The Pile)

Max Activating Token Index: 162

Text #15

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.3250. Min Act: -0.0000

Data Index: 1295396 (The Pile)

Max Activating Token Index: 754

Text #16

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.3358. Min Act: -0.0000

Data Index: 215409 (The Pile)

Max Activating Token Index: 882

Text #17

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.1288. Min Act: -0.0000

Data Index: 693674 (The Pile)

Max Activating Token Index: 980

Text #18

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.5024. Min Act: -0.0000

Data Index: 1545032 (The Pile)

Max Activating Token Index: 360

Text #19

Max Range: 3.7596. Min Range: -3.7596

Max Act: 2.2443. Min Act: -0.0000

Data Index: 1936388 (The Pile)

Max Activating Token Index: 372