Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2761 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 12.3948. Min Range: -12.3948

Max Act: 12.3948. Min Act: -0.0000

Data Index: 1245425 (The Pile)

Max Activating Token Index: 118

Text #1

Max Range: 12.3948. Min Range: -12.3948

Max Act: 12.2960. Min Act: -0.0000

Data Index: 1828291 (The Pile)

Max Activating Token Index: 80

Text #2

Max Range: 12.3948. Min Range: -12.3948

Max Act: 11.7614. Min Act: -0.0000

Data Index: 1089310 (The Pile)

Max Activating Token Index: 121

Text #3

Max Range: 12.3948. Min Range: -12.3948

Max Act: 11.2681. Min Act: -0.0000

Data Index: 46898 (The Pile)

Max Activating Token Index: 40

Text #4

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.9292. Min Act: -0.0000

Data Index: 707923 (The Pile)

Max Activating Token Index: 229

Text #5

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.8739. Min Act: -0.0000

Data Index: 1543229 (The Pile)

Max Activating Token Index: 272

Text #6

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.4807. Min Act: -0.0000

Data Index: 1897084 (The Pile)

Max Activating Token Index: 36

Text #7

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.4347. Min Act: -0.0000

Data Index: 1256650 (The Pile)

Max Activating Token Index: 722

Text #8

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.3488. Min Act: -0.0000

Data Index: 743686 (The Pile)

Max Activating Token Index: 80

Text #9

Max Range: 12.3948. Min Range: -12.3948

Max Act: 9.5623. Min Act: -0.0000

Data Index: 1739921 (The Pile)

Max Activating Token Index: 15

Text #10

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.0080. Min Act: -0.0000

Data Index: 86347 (The Pile)

Max Activating Token Index: 235

Text #11

Max Range: 12.3948. Min Range: -12.3948

Max Act: 9.9571. Min Act: -0.0000

Data Index: 997699 (The Pile)

Max Activating Token Index: 280

Text #12

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.0662. Min Act: -0.0000

Data Index: 789908 (The Pile)

Max Activating Token Index: 38

Text #13

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.3261. Min Act: -0.0000

Data Index: 1345449 (The Pile)

Max Activating Token Index: 51

Text #14

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.2745. Min Act: -0.0000

Data Index: 629925 (The Pile)

Max Activating Token Index: 39

Text #15

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.0973. Min Act: -0.0000

Data Index: 1990410 (The Pile)

Max Activating Token Index: 64

Text #16

Max Range: 12.3948. Min Range: -12.3948

Max Act: 9.9800. Min Act: -0.0000

Data Index: 1419630 (The Pile)

Max Activating Token Index: 27

Text #17

Max Range: 12.3948. Min Range: -12.3948

Max Act: 9.2669. Min Act: -0.0000

Data Index: 977005 (The Pile)

Max Activating Token Index: 137

Text #18

Max Range: 12.3948. Min Range: -12.3948

Max Act: 9.2807. Min Act: -0.0000

Data Index: 240582 (The Pile)

Max Activating Token Index: 83

Text #19

Max Range: 12.3948. Min Range: -12.3948

Max Act: 10.0550. Min Act: -0.0000

Data Index: 1223448 (The Pile)

Max Activating Token Index: 56