Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3401 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.9378. Min Act: -0.0000

Data Index: 743778 (The Pile)

Max Activating Token Index: 481

Text #1

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.9081. Min Act: -0.0000

Data Index: 1668496 (The Pile)

Max Activating Token Index: 349

Text #2

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.6249. Min Act: -0.0000

Data Index: 1889861 (The Pile)

Max Activating Token Index: 856

Text #3

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.6935. Min Act: -0.0000

Data Index: 665323 (The Pile)

Max Activating Token Index: 271

Text #4

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5174. Min Act: -0.0000

Data Index: 1283271 (The Pile)

Max Activating Token Index: 679

Text #5

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.6327. Min Act: -0.0000

Data Index: 1329128 (The Pile)

Max Activating Token Index: 444

Text #6

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5894. Min Act: -0.0000

Data Index: 290473 (The Pile)

Max Activating Token Index: 759

Text #7

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.6243. Min Act: -0.0000

Data Index: 11620 (The Pile)

Max Activating Token Index: 609

Text #8

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.6257. Min Act: -0.0000

Data Index: 1127093 (The Pile)

Max Activating Token Index: 275

Text #9

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5717. Min Act: -0.0000

Data Index: 1845704 (The Pile)

Max Activating Token Index: 268

Text #10

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.4299. Min Act: -0.0000

Data Index: 929622 (The Pile)

Max Activating Token Index: 694

Text #11

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5074. Min Act: -0.0000

Data Index: 1533370 (The Pile)

Max Activating Token Index: 622

Text #12

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.4321. Min Act: -0.0000

Data Index: 1589727 (The Pile)

Max Activating Token Index: 698

Text #13

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5744. Min Act: -0.0000

Data Index: 730248 (The Pile)

Max Activating Token Index: 95

Text #14

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.4417. Min Act: -0.0000

Data Index: 535778 (The Pile)

Max Activating Token Index: 293

Text #15

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.4906. Min Act: -0.0000

Data Index: 279987 (The Pile)

Max Activating Token Index: 433

Text #16

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5344. Min Act: -0.0000

Data Index: 1420691 (The Pile)

Max Activating Token Index: 973

Text #17

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5509. Min Act: -0.0000

Data Index: 106593 (The Pile)

Max Activating Token Index: 322

Text #18

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5074. Min Act: -0.0000

Data Index: 1702007 (The Pile)

Max Activating Token Index: 154

Text #19

Max Range: 1.9378. Min Range: -1.9378

Max Act: 1.5157. Min Act: -0.0000

Data Index: 1565644 (The Pile)

Max Activating Token Index: 368