Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1415 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.8760. Min Act: -0.0000

Data Index: 857696 (The Pile)

Max Activating Token Index: 44

Text #1

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.8104. Min Act: -0.0000

Data Index: 1525962 (The Pile)

Max Activating Token Index: 119

Text #2

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.6787. Min Act: -0.0000

Data Index: 1206464 (The Pile)

Max Activating Token Index: 88

Text #3

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5264. Min Act: -0.0000

Data Index: 784841 (The Pile)

Max Activating Token Index: 150

Text #4

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.6172. Min Act: 0.0000

Data Index: 293780 (The Pile)

Max Activating Token Index: 600

Text #5

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5982. Min Act: 0.0000

Data Index: 880405 (The Pile)

Max Activating Token Index: 289

Text #6

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5310. Min Act: -0.0000

Data Index: 271413 (The Pile)

Max Activating Token Index: 839

Text #7

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5609. Min Act: 0.0000

Data Index: 234551 (The Pile)

Max Activating Token Index: 109

Text #8

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.6103. Min Act: 0.0000

Data Index: 1654018 (The Pile)

Max Activating Token Index: 52

Text #9

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5285. Min Act: 0.0001

Data Index: 275487 (The Pile)

Max Activating Token Index: 438

Text #10

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4426. Min Act: 0.0001

Data Index: 1221172 (The Pile)

Max Activating Token Index: 289

Text #11

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4081. Min Act: 0.0000

Data Index: 1256377 (The Pile)

Max Activating Token Index: 832

Text #12

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4610. Min Act: 0.0000

Data Index: 1105810 (The Pile)

Max Activating Token Index: 617

Text #13

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4757. Min Act: -0.0000

Data Index: 227063 (The Pile)

Max Activating Token Index: 89

Text #14

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4656. Min Act: -0.0000

Data Index: 927605 (The Pile)

Max Activating Token Index: 718

Text #15

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4394. Min Act: -0.0000

Data Index: 1157549 (The Pile)

Max Activating Token Index: 102

Text #16

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5382. Min Act: -0.0000

Data Index: 660766 (The Pile)

Max Activating Token Index: 493

Text #17

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.5369. Min Act: -0.0000

Data Index: 1788030 (The Pile)

Max Activating Token Index: 93

Text #18

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.3607. Min Act: -0.0000

Data Index: 561077 (The Pile)

Max Activating Token Index: 822

Text #19

Max Range: 1.8760. Min Range: -1.8760

Max Act: 1.4256. Min Act: 0.0000

Data Index: 829060 (The Pile)

Max Activating Token Index: 63