Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4397 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.7430. Min Act: -0.0000

Data Index: 270014 (The Pile)

Max Activating Token Index: 41

Text #1

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6937. Min Act: -0.0000

Data Index: 1003931 (The Pile)

Max Activating Token Index: 309

Text #2

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6863. Min Act: -0.0000

Data Index: 1506347 (The Pile)

Max Activating Token Index: 124

Text #3

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6199. Min Act: -0.0000

Data Index: 188220 (The Pile)

Max Activating Token Index: 217

Text #4

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6350. Min Act: -0.0000

Data Index: 1184623 (The Pile)

Max Activating Token Index: 156

Text #5

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6249. Min Act: -0.0000

Data Index: 1974245 (The Pile)

Max Activating Token Index: 20

Text #6

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6133. Min Act: -0.0000

Data Index: 1851883 (The Pile)

Max Activating Token Index: 154

Text #7

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6020. Min Act: -0.0000

Data Index: 73511 (The Pile)

Max Activating Token Index: 336

Text #8

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.5671. Min Act: -0.0000

Data Index: 667064 (The Pile)

Max Activating Token Index: 40

Text #9

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6184. Min Act: -0.0000

Data Index: 1125572 (The Pile)

Max Activating Token Index: 833

Text #10

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.6139. Min Act: -0.0000

Data Index: 1984839 (The Pile)

Max Activating Token Index: 280

Text #11

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.5713. Min Act: -0.0000

Data Index: 1531980 (The Pile)

Max Activating Token Index: 54

Text #12

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.5480. Min Act: -0.0000

Data Index: 375898 (The Pile)

Max Activating Token Index: 179

Text #13

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4939. Min Act: -0.0000

Data Index: 1963604 (The Pile)

Max Activating Token Index: 168

Text #14

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4790. Min Act: -0.0000

Data Index: 1778620 (The Pile)

Max Activating Token Index: 605

Text #15

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4978. Min Act: -0.0000

Data Index: 1276936 (The Pile)

Max Activating Token Index: 36

Text #16

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4772. Min Act: -0.0000

Data Index: 244438 (The Pile)

Max Activating Token Index: 811

Text #17

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4977. Min Act: -0.0000

Data Index: 1054715 (The Pile)

Max Activating Token Index: 189

Text #18

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4806. Min Act: -0.0000

Data Index: 795240 (The Pile)

Max Activating Token Index: 485

Text #19

Max Range: 0.7430. Min Range: -0.7430

Max Act: 0.4670. Min Act: -0.0000

Data Index: 1716748 (The Pile)

Max Activating Token Index: 200