Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3422 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.6504. Min Act: -0.0000

Data Index: 10170 (The Pile)

Max Activating Token Index: 108

Text #1

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.4117. Min Act: -0.0000

Data Index: 562725 (The Pile)

Max Activating Token Index: 527

Text #2

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.5241. Min Act: -0.0000

Data Index: 1454960 (The Pile)

Max Activating Token Index: 252

Text #3

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.5296. Min Act: -0.0000

Data Index: 1041340 (The Pile)

Max Activating Token Index: 738

Text #4

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.3503. Min Act: -0.0000

Data Index: 568235 (The Pile)

Max Activating Token Index: 618

Text #5

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.3408. Min Act: -0.0001

Data Index: 141468 (The Pile)

Max Activating Token Index: 83

Text #6

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.3084. Min Act: -0.0000

Data Index: 673367 (The Pile)

Max Activating Token Index: 882

Text #7

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.2652. Min Act: -0.0000

Data Index: 1974611 (The Pile)

Max Activating Token Index: 199

Text #8

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0826. Min Act: -0.0000

Data Index: 972600 (The Pile)

Max Activating Token Index: 592

Text #9

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.1783. Min Act: -0.0000

Data Index: 628968 (The Pile)

Max Activating Token Index: 265

Text #10

Max Range: 3.6504. Min Range: -3.6504

Max Act: 2.9825. Min Act: -0.0000

Data Index: 1091760 (The Pile)

Max Activating Token Index: 955

Text #11

Max Range: 3.6504. Min Range: -3.6504

Max Act: 2.9866. Min Act: -0.0000

Data Index: 1006283 (The Pile)

Max Activating Token Index: 614

Text #12

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0837. Min Act: -0.0000

Data Index: 873105 (The Pile)

Max Activating Token Index: 263

Text #13

Max Range: 3.6504. Min Range: -3.6504

Max Act: 2.9475. Min Act: -0.0000

Data Index: 616133 (The Pile)

Max Activating Token Index: 61

Text #14

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0889. Min Act: -0.0000

Data Index: 519827 (The Pile)

Max Activating Token Index: 857

Text #15

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0641. Min Act: -0.0000

Data Index: 665084 (The Pile)

Max Activating Token Index: 965

Text #16

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0629. Min Act: -0.0000

Data Index: 426750 (The Pile)

Max Activating Token Index: 594

Text #17

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.1676. Min Act: -0.0000

Data Index: 1212421 (The Pile)

Max Activating Token Index: 241

Text #18

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.0348. Min Act: -0.0000

Data Index: 1607999 (The Pile)

Max Activating Token Index: 167

Text #19

Max Range: 3.6504. Min Range: -3.6504

Max Act: 3.1840. Min Act: -0.0000

Data Index: 1005503 (The Pile)

Max Activating Token Index: 864