Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2317 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.4331. Min Act: -0.0000

Data Index: 787015 (The Pile)

Max Activating Token Index: 37

Text #1

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.9249. Min Act: -0.0000

Data Index: 618449 (The Pile)

Max Activating Token Index: 389

Text #2

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.6368. Min Act: -0.0000

Data Index: 821122 (The Pile)

Max Activating Token Index: 103

Text #3

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.4315. Min Act: -0.0000

Data Index: 169627 (The Pile)

Max Activating Token Index: 318

Text #4

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.7262. Min Act: -0.0000

Data Index: 1258068 (The Pile)

Max Activating Token Index: 489

Text #5

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.3481. Min Act: -0.0000

Data Index: 1538086 (The Pile)

Max Activating Token Index: 115

Text #6

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.5217. Min Act: -0.0000

Data Index: 1493335 (The Pile)

Max Activating Token Index: 66

Text #7

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.4579. Min Act: -0.0000

Data Index: 840241 (The Pile)

Max Activating Token Index: 123

Text #8

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.2767. Min Act: -0.0000

Data Index: 930452 (The Pile)

Max Activating Token Index: 146

Text #9

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.2579. Min Act: -0.0000

Data Index: 1712974 (The Pile)

Max Activating Token Index: 65

Text #10

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.2297. Min Act: -0.0000

Data Index: 929041 (The Pile)

Max Activating Token Index: 5

Text #11

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.5295. Min Act: -0.0000

Data Index: 1364315 (The Pile)

Max Activating Token Index: 38

Text #12

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.3780. Min Act: -0.0000

Data Index: 1854268 (The Pile)

Max Activating Token Index: 81

Text #13

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.4611. Min Act: -0.0000

Data Index: 1693988 (The Pile)

Max Activating Token Index: 141

Text #14

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.1637. Min Act: -0.0000

Data Index: 674803 (The Pile)

Max Activating Token Index: 726

Text #15

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.3098. Min Act: -0.0000

Data Index: 1321933 (The Pile)

Max Activating Token Index: 428

Text #16

Max Range: 3.9249. Min Range: -3.9249

Max Act: 2.9766. Min Act: -0.0000

Data Index: 459309 (The Pile)

Max Activating Token Index: 230

Text #17

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.1604. Min Act: -0.0000

Data Index: 1016495 (The Pile)

Max Activating Token Index: 285

Text #18

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.4718. Min Act: -0.0000

Data Index: 592716 (The Pile)

Max Activating Token Index: 11

Text #19

Max Range: 3.9249. Min Range: -3.9249

Max Act: 3.1076. Min Act: -0.0000

Data Index: 485426 (The Pile)

Max Activating Token Index: 683