Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4838 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.0376. Min Range: -3.0376

Max Act: 3.0187. Min Act: -0.0000

Data Index: 1829834 (The Pile)

Max Activating Token Index: 848

Text #1

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.8204. Min Act: -0.0000

Data Index: 875800 (The Pile)

Max Activating Token Index: 430

Text #2

Max Range: 3.0376. Min Range: -3.0376

Max Act: 3.0376. Min Act: -0.0000

Data Index: 1048754 (The Pile)

Max Activating Token Index: 745

Text #3

Max Range: 3.0376. Min Range: -3.0376

Max Act: 1.9708. Min Act: -0.0000

Data Index: 1086778 (The Pile)

Max Activating Token Index: 829

Text #4

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.6467. Min Act: -0.0000

Data Index: 326816 (The Pile)

Max Activating Token Index: 740

Text #5

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.4020. Min Act: -0.0000

Data Index: 758694 (The Pile)

Max Activating Token Index: 69

Text #6

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.2434. Min Act: -0.0000

Data Index: 1920936 (The Pile)

Max Activating Token Index: 276

Text #7

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.0419. Min Act: -0.0000

Data Index: 829407 (The Pile)

Max Activating Token Index: 461

Text #8

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.1735. Min Act: -0.0000

Data Index: 1152455 (The Pile)

Max Activating Token Index: 325

Text #9

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.2592. Min Act: -0.0000

Data Index: 1485264 (The Pile)

Max Activating Token Index: 744

Text #10

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.3495. Min Act: -0.0000

Data Index: 230299 (The Pile)

Max Activating Token Index: 902

Text #11

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.1632. Min Act: -0.0000

Data Index: 65159 (The Pile)

Max Activating Token Index: 46

Text #12

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.0203. Min Act: -0.0000

Data Index: 415672 (The Pile)

Max Activating Token Index: 69

Text #13

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.0349. Min Act: -0.0000

Data Index: 918263 (The Pile)

Max Activating Token Index: 426

Text #14

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.2848. Min Act: -0.0000

Data Index: 25592 (The Pile)

Max Activating Token Index: 198

Text #15

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.0173. Min Act: -0.0000

Data Index: 1146214 (The Pile)

Max Activating Token Index: 862

Text #16

Max Range: 3.0376. Min Range: -3.0376

Max Act: 2.1188. Min Act: -0.0000

Data Index: 1573609 (The Pile)

Max Activating Token Index: 533

Text #17

Max Range: 3.0376. Min Range: -3.0376

Max Act: 1.9938. Min Act: -0.0000

Data Index: 1090309 (The Pile)

Max Activating Token Index: 601

Text #18

Max Range: 3.0376. Min Range: -3.0376

Max Act: 1.9379. Min Act: -0.0000

Data Index: 262424 (The Pile)

Max Activating Token Index: 800

Text #19

Max Range: 3.0376. Min Range: -3.0376

Max Act: 1.8955. Min Act: -0.0000

Data Index: 1717323 (The Pile)

Max Activating Token Index: 301