Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 82 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.7856. Min Act: -0.0000

Data Index: 1595896 (The Pile)

Max Activating Token Index: 89

Text #1

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.8799. Min Act: -0.0000

Data Index: 184141 (The Pile)

Max Activating Token Index: 166

Text #2

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.6243. Min Act: -0.0000

Data Index: 302190 (The Pile)

Max Activating Token Index: 62

Text #3

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.4821. Min Act: -0.0000

Data Index: 1993475 (The Pile)

Max Activating Token Index: 687

Text #4

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.4534. Min Act: -0.0000

Data Index: 295244 (The Pile)

Max Activating Token Index: 597

Text #5

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.3576. Min Act: -0.0000

Data Index: 837612 (The Pile)

Max Activating Token Index: 880

Text #6

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.1862. Min Act: -0.0000

Data Index: 313831 (The Pile)

Max Activating Token Index: 711

Text #7

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.0751. Min Act: -0.0000

Data Index: 1950130 (The Pile)

Max Activating Token Index: 244

Text #8

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.3209. Min Act: -0.0000

Data Index: 1244798 (The Pile)

Max Activating Token Index: 228

Text #9

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.2456. Min Act: -0.0000

Data Index: 513395 (The Pile)

Max Activating Token Index: 23

Text #10

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.3583. Min Act: -0.0000

Data Index: 1373624 (The Pile)

Max Activating Token Index: 741

Text #11

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.2489. Min Act: -0.0000

Data Index: 1020467 (The Pile)

Max Activating Token Index: 939

Text #12

Max Range: 1.8799. Min Range: -1.8799

Max Act: 0.9972. Min Act: -0.0000

Data Index: 1604461 (The Pile)

Max Activating Token Index: 922

Text #13

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.1613. Min Act: -0.0000

Data Index: 1197051 (The Pile)

Max Activating Token Index: 485

Text #14

Max Range: 1.8799. Min Range: -1.8799

Max Act: 0.9553. Min Act: -0.0000

Data Index: 1308820 (The Pile)

Max Activating Token Index: 329

Text #15

Max Range: 1.8799. Min Range: -1.8799

Max Act: 0.9061. Min Act: -0.0000

Data Index: 1049095 (The Pile)

Max Activating Token Index: 590

Text #16

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.0790. Min Act: -0.0000

Data Index: 1345629 (The Pile)

Max Activating Token Index: 494

Text #17

Max Range: 1.8799. Min Range: -1.8799

Max Act: 0.9392. Min Act: -0.0000

Data Index: 1525391 (The Pile)

Max Activating Token Index: 869

Text #18

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.0807. Min Act: -0.0000

Data Index: 790170 (The Pile)

Max Activating Token Index: 876

Text #19

Max Range: 1.8799. Min Range: -1.8799

Max Act: 1.0093. Min Act: -0.0000

Data Index: 60777 (The Pile)

Max Activating Token Index: 422