Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 316 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.8641. Min Act: -0.0000

Data Index: 652807 (The Pile)

Max Activating Token Index: 73

Text #1

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.7789. Min Act: -0.0000

Data Index: 1942976 (The Pile)

Max Activating Token Index: 161

Text #2

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6961. Min Act: -0.0000

Data Index: 1593569 (The Pile)

Max Activating Token Index: 516

Text #3

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.7918. Min Act: -0.0000

Data Index: 1224745 (The Pile)

Max Activating Token Index: 97

Text #4

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6401. Min Act: -0.0000

Data Index: 1989509 (The Pile)

Max Activating Token Index: 871

Text #5

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6748. Min Act: -0.0000

Data Index: 1260672 (The Pile)

Max Activating Token Index: 80

Text #6

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.8352. Min Act: -0.0000

Data Index: 994332 (The Pile)

Max Activating Token Index: 85

Text #7

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.5693. Min Act: -0.0000

Data Index: 1530879 (The Pile)

Max Activating Token Index: 530

Text #8

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.9179. Min Act: -0.0000

Data Index: 1129406 (The Pile)

Max Activating Token Index: 745

Text #9

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6274. Min Act: -0.0000

Data Index: 756071 (The Pile)

Max Activating Token Index: 406

Text #10

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6500. Min Act: -0.0000

Data Index: 1557265 (The Pile)

Max Activating Token Index: 663

Text #11

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.5438. Min Act: -0.0000

Data Index: 274409 (The Pile)

Max Activating Token Index: 675

Text #12

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6367. Min Act: -0.0000

Data Index: 223102 (The Pile)

Max Activating Token Index: 646

Text #13

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.5323. Min Act: -0.0000

Data Index: 832506 (The Pile)

Max Activating Token Index: 549

Text #14

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.5564. Min Act: -0.0000

Data Index: 74114 (The Pile)

Max Activating Token Index: 237

Text #15

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.5375. Min Act: -0.0000

Data Index: 848593 (The Pile)

Max Activating Token Index: 886

Text #16

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6508. Min Act: -0.0000

Data Index: 1893914 (The Pile)

Max Activating Token Index: 971

Text #17

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.4923. Min Act: 0.0000

Data Index: 1542160 (The Pile)

Max Activating Token Index: 164

Text #18

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6089. Min Act: -0.0000

Data Index: 302005 (The Pile)

Max Activating Token Index: 676

Text #19

Max Range: 1.9179. Min Range: -1.9179

Max Act: 1.6224. Min Act: 0.0000

Data Index: 1685614 (The Pile)

Max Activating Token Index: 736