Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3763 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.8378. Min Act: -0.0000

Data Index: 164417 (The Pile)

Max Activating Token Index: 464

Text #1

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.5222. Min Act: -0.0000

Data Index: 1056090 (The Pile)

Max Activating Token Index: 372

Text #2

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.5342. Min Act: -0.0000

Data Index: 953034 (The Pile)

Max Activating Token Index: 586

Text #3

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.5813. Min Act: -0.0000

Data Index: 238778 (The Pile)

Max Activating Token Index: 133

Text #4

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4052. Min Act: -0.0000

Data Index: 839226 (The Pile)

Max Activating Token Index: 668

Text #5

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4772. Min Act: -0.0000

Data Index: 743723 (The Pile)

Max Activating Token Index: 370

Text #6

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4521. Min Act: -0.0000

Data Index: 1662969 (The Pile)

Max Activating Token Index: 137

Text #7

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4592. Min Act: -0.0000

Data Index: 2941 (The Pile)

Max Activating Token Index: 777

Text #8

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4502. Min Act: -0.0000

Data Index: 675016 (The Pile)

Max Activating Token Index: 481

Text #9

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4295. Min Act: -0.0000

Data Index: 1962821 (The Pile)

Max Activating Token Index: 240

Text #10

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.4170. Min Act: -0.0000

Data Index: 1927379 (The Pile)

Max Activating Token Index: 238

Text #11

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3908. Min Act: -0.0000

Data Index: 245216 (The Pile)

Max Activating Token Index: 276

Text #12

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3416. Min Act: -0.0000

Data Index: 290348 (The Pile)

Max Activating Token Index: 655

Text #13

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3792. Min Act: -0.0000

Data Index: 539643 (The Pile)

Max Activating Token Index: 432

Text #14

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3607. Min Act: -0.0000

Data Index: 534126 (The Pile)

Max Activating Token Index: 196

Text #15

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3362. Min Act: -0.0000

Data Index: 1710286 (The Pile)

Max Activating Token Index: 783

Text #16

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.2621. Min Act: -0.0000

Data Index: 1890662 (The Pile)

Max Activating Token Index: 634

Text #17

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.2403. Min Act: -0.0000

Data Index: 1116539 (The Pile)

Max Activating Token Index: 740

Text #18

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3231. Min Act: -0.0000

Data Index: 279540 (The Pile)

Max Activating Token Index: 758

Text #19

Max Range: 1.8378. Min Range: -1.8378

Max Act: 1.3955. Min Act: -0.0000

Data Index: 1034761 (The Pile)

Max Activating Token Index: 697