Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2215 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.7597. Min Act: -0.0000

Data Index: 1920523 (The Pile)

Max Activating Token Index: 224

Text #1

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.7273. Min Act: -0.0000

Data Index: 479544 (The Pile)

Max Activating Token Index: 453

Text #2

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.7040. Min Act: -0.0000

Data Index: 1366665 (The Pile)

Max Activating Token Index: 786

Text #3

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6937. Min Act: -0.0000

Data Index: 1254603 (The Pile)

Max Activating Token Index: 198

Text #4

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6594. Min Act: -0.0000

Data Index: 52871 (The Pile)

Max Activating Token Index: 881

Text #5

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5638. Min Act: -0.0000

Data Index: 393304 (The Pile)

Max Activating Token Index: 488

Text #6

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5529. Min Act: -0.0000

Data Index: 1438453 (The Pile)

Max Activating Token Index: 795

Text #7

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5773. Min Act: -0.0000

Data Index: 1479538 (The Pile)

Max Activating Token Index: 492

Text #8

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5445. Min Act: -0.0000

Data Index: 477432 (The Pile)

Max Activating Token Index: 596

Text #9

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5300. Min Act: -0.0000

Data Index: 210383 (The Pile)

Max Activating Token Index: 912

Text #10

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5241. Min Act: -0.0000

Data Index: 1640675 (The Pile)

Max Activating Token Index: 794

Text #11

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5403. Min Act: -0.0000

Data Index: 1918049 (The Pile)

Max Activating Token Index: 140

Text #12

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.4993. Min Act: -0.0000

Data Index: 127483 (The Pile)

Max Activating Token Index: 676

Text #13

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5561. Min Act: -0.0000

Data Index: 998960 (The Pile)

Max Activating Token Index: 370

Text #14

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5370. Min Act: -0.0000

Data Index: 799543 (The Pile)

Max Activating Token Index: 757

Text #15

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5445. Min Act: -0.0000

Data Index: 976231 (The Pile)

Max Activating Token Index: 1018

Text #16

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5400. Min Act: -0.0000

Data Index: 1396854 (The Pile)

Max Activating Token Index: 532

Text #17

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5406. Min Act: -0.0000

Data Index: 1063317 (The Pile)

Max Activating Token Index: 466

Text #18

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5323. Min Act: -0.0000

Data Index: 125763 (The Pile)

Max Activating Token Index: 962

Text #19

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5348. Min Act: -0.0000

Data Index: 795516 (The Pile)

Max Activating Token Index: 675