Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4646 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.4195. Min Act: -0.0000

Data Index: 134113 (The Pile)

Max Activating Token Index: 480

Text #1

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.6701. Min Act: -0.0000

Data Index: 897674 (The Pile)

Max Activating Token Index: 378

Text #2

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.5069. Min Act: -0.0000

Data Index: 554733 (The Pile)

Max Activating Token Index: 693

Text #3

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.9583. Min Act: -0.0000

Data Index: 110207 (The Pile)

Max Activating Token Index: 935

Text #4

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.1219. Min Act: 0.0000

Data Index: 1128504 (The Pile)

Max Activating Token Index: 517

Text #5

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.2971. Min Act: -0.0000

Data Index: 818017 (The Pile)

Max Activating Token Index: 129

Text #6

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.9672. Min Act: -0.0000

Data Index: 516695 (The Pile)

Max Activating Token Index: 974

Text #7

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.9595. Min Act: -0.0000

Data Index: 1890568 (The Pile)

Max Activating Token Index: 140

Text #8

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.0598. Min Act: -0.0000

Data Index: 1035467 (The Pile)

Max Activating Token Index: 375

Text #9

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.3510. Min Act: -0.0000

Data Index: 482340 (The Pile)

Max Activating Token Index: 297

Text #10

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.1606. Min Act: -0.0000

Data Index: 629109 (The Pile)

Max Activating Token Index: 680

Text #11

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.0006. Min Act: -0.0000

Data Index: 972641 (The Pile)

Max Activating Token Index: 545

Text #12

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.0211. Min Act: -0.0000

Data Index: 1510546 (The Pile)

Max Activating Token Index: 496

Text #13

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.7491. Min Act: -0.0000

Data Index: 1238831 (The Pile)

Max Activating Token Index: 894

Text #14

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.8602. Min Act: -0.0000

Data Index: 1321093 (The Pile)

Max Activating Token Index: 33

Text #15

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.9947. Min Act: -0.0000

Data Index: 976261 (The Pile)

Max Activating Token Index: 30

Text #16

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.9164. Min Act: -0.0000

Data Index: 1538159 (The Pile)

Max Activating Token Index: 367

Text #17

Max Range: 3.6701. Min Range: -3.6701

Max Act: 3.0513. Min Act: -0.0000

Data Index: 1512978 (The Pile)

Max Activating Token Index: 834

Text #18

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.8838. Min Act: -0.0000

Data Index: 585801 (The Pile)

Max Activating Token Index: 481

Text #19

Max Range: 3.6701. Min Range: -3.6701

Max Act: 2.8074. Min Act: -0.0000

Data Index: 290816 (The Pile)

Max Activating Token Index: 730