Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3167 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7695. Min Act: -0.0000

Data Index: 570866 (The Pile)

Max Activating Token Index: 486

Text #1

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6518. Min Act: -0.0000

Data Index: 1178730 (The Pile)

Max Activating Token Index: 621

Text #2

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7554. Min Act: -0.0000

Data Index: 801258 (The Pile)

Max Activating Token Index: 774

Text #3

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6909. Min Act: -0.0000

Data Index: 950235 (The Pile)

Max Activating Token Index: 268

Text #4

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7546. Min Act: -0.0000

Data Index: 427370 (The Pile)

Max Activating Token Index: 714

Text #5

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7097. Min Act: -0.0000

Data Index: 1792912 (The Pile)

Max Activating Token Index: 212

Text #6

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6675. Min Act: -0.0000

Data Index: 1011669 (The Pile)

Max Activating Token Index: 507

Text #7

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7611. Min Act: 0.0000

Data Index: 288891 (The Pile)

Max Activating Token Index: 934

Text #8

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6734. Min Act: -0.0000

Data Index: 43291 (The Pile)

Max Activating Token Index: 158

Text #9

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6980. Min Act: -0.0000

Data Index: 662419 (The Pile)

Max Activating Token Index: 402

Text #10

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.7483. Min Act: -0.0000

Data Index: 1736388 (The Pile)

Max Activating Token Index: 177

Text #11

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6887. Min Act: -0.0000

Data Index: 990903 (The Pile)

Max Activating Token Index: 552

Text #12

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6828. Min Act: -0.0000

Data Index: 418085 (The Pile)

Max Activating Token Index: 129

Text #13

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6978. Min Act: -0.0000

Data Index: 200505 (The Pile)

Max Activating Token Index: 391

Text #14

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6243. Min Act: -0.0000

Data Index: 427685 (The Pile)

Max Activating Token Index: 491

Text #15

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6055. Min Act: -0.0000

Data Index: 1069513 (The Pile)

Max Activating Token Index: 183

Text #16

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6647. Min Act: -0.0000

Data Index: 897355 (The Pile)

Max Activating Token Index: 113

Text #17

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6559. Min Act: 0.0000

Data Index: 570044 (The Pile)

Max Activating Token Index: 1022

Text #18

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.6086. Min Act: -0.0000

Data Index: 1182046 (The Pile)

Max Activating Token Index: 429

Text #19

Max Range: 0.7695. Min Range: -0.7695

Max Act: 0.5704. Min Act: -0.0000

Data Index: 923807 (The Pile)

Max Activating Token Index: 1013