Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 5068 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.8403. Min Act: -0.0000

Data Index: 716777 (The Pile)

Max Activating Token Index: 993

Text #1

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.8209. Min Act: -0.0000

Data Index: 631945 (The Pile)

Max Activating Token Index: 288

Text #2

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.7241. Min Act: -0.0000

Data Index: 812544 (The Pile)

Max Activating Token Index: 379

Text #3

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6995. Min Act: -0.0000

Data Index: 1554022 (The Pile)

Max Activating Token Index: 842

Text #4

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6365. Min Act: -0.0000

Data Index: 914148 (The Pile)

Max Activating Token Index: 30

Text #5

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6204. Min Act: -0.0000

Data Index: 1441796 (The Pile)

Max Activating Token Index: 87

Text #6

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5988. Min Act: -0.0000

Data Index: 471100 (The Pile)

Max Activating Token Index: 141

Text #7

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6276. Min Act: -0.0000

Data Index: 770339 (The Pile)

Max Activating Token Index: 108

Text #8

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6018. Min Act: -0.0000

Data Index: 684445 (The Pile)

Max Activating Token Index: 734

Text #9

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6657. Min Act: -0.0000

Data Index: 704268 (The Pile)

Max Activating Token Index: 581

Text #10

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6302. Min Act: -0.0000

Data Index: 1522644 (The Pile)

Max Activating Token Index: 502

Text #11

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6441. Min Act: -0.0000

Data Index: 1577356 (The Pile)

Max Activating Token Index: 681

Text #12

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5972. Min Act: -0.0000

Data Index: 1434937 (The Pile)

Max Activating Token Index: 267

Text #13

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6201. Min Act: -0.0000

Data Index: 523905 (The Pile)

Max Activating Token Index: 395

Text #14

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5753. Min Act: -0.0000

Data Index: 1113080 (The Pile)

Max Activating Token Index: 35

Text #15

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5828. Min Act: -0.0000

Data Index: 1355439 (The Pile)

Max Activating Token Index: 1020

Text #16

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5836. Min Act: -0.0000

Data Index: 1606539 (The Pile)

Max Activating Token Index: 100

Text #17

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.5382. Min Act: -0.0000

Data Index: 1910003 (The Pile)

Max Activating Token Index: 693

Text #18

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6486. Min Act: -0.0000

Data Index: 560271 (The Pile)

Max Activating Token Index: 869

Text #19

Max Range: 0.8403. Min Range: -0.8403

Max Act: 0.6063. Min Act: -0.0000

Data Index: 390278 (The Pile)

Max Activating Token Index: 470