Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3722 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.8675. Min Act: -0.0000

Data Index: 694355 (The Pile)

Max Activating Token Index: 1018

Text #1

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7805. Min Act: -0.0000

Data Index: 343696 (The Pile)

Max Activating Token Index: 964

Text #2

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.8117. Min Act: -0.0000

Data Index: 415465 (The Pile)

Max Activating Token Index: 344

Text #3

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6819. Min Act: -0.0000

Data Index: 363410 (The Pile)

Max Activating Token Index: 1012

Text #4

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7254. Min Act: -0.0000

Data Index: 963887 (The Pile)

Max Activating Token Index: 433

Text #5

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7276. Min Act: -0.0000

Data Index: 1879587 (The Pile)

Max Activating Token Index: 123

Text #6

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7137. Min Act: -0.0000

Data Index: 720844 (The Pile)

Max Activating Token Index: 297

Text #7

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7145. Min Act: -0.0000

Data Index: 265921 (The Pile)

Max Activating Token Index: 434

Text #8

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7089. Min Act: -0.0000

Data Index: 407784 (The Pile)

Max Activating Token Index: 789

Text #9

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6769. Min Act: -0.0000

Data Index: 1366570 (The Pile)

Max Activating Token Index: 509

Text #10

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7159. Min Act: -0.0000

Data Index: 426049 (The Pile)

Max Activating Token Index: 933

Text #11

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6755. Min Act: -0.0000

Data Index: 484924 (The Pile)

Max Activating Token Index: 160

Text #12

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7311. Min Act: -0.0000

Data Index: 1340788 (The Pile)

Max Activating Token Index: 983

Text #13

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6700. Min Act: -0.0000

Data Index: 1347439 (The Pile)

Max Activating Token Index: 454

Text #14

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6463. Min Act: -0.0000

Data Index: 1211436 (The Pile)

Max Activating Token Index: 287

Text #15

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.7176. Min Act: -0.0000

Data Index: 197501 (The Pile)

Max Activating Token Index: 682

Text #16

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6671. Min Act: -0.0000

Data Index: 1290011 (The Pile)

Max Activating Token Index: 333

Text #17

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6632. Min Act: -0.0000

Data Index: 1649899 (The Pile)

Max Activating Token Index: 768

Text #18

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6868. Min Act: -0.0000

Data Index: 1938232 (The Pile)

Max Activating Token Index: 266

Text #19

Max Range: 0.8675. Min Range: -0.8675

Max Act: 0.6399. Min Act: -0.0000

Data Index: 1597383 (The Pile)

Max Activating Token Index: 339