Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2930 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.9951. Min Act: -0.0000

Data Index: 1830379 (The Pile)

Max Activating Token Index: 907

Text #1

Max Range: 1.0459. Min Range: -1.0459

Max Act: 1.0459. Min Act: -0.0000

Data Index: 1721656 (The Pile)

Max Activating Token Index: 755

Text #2

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.9673. Min Act: -0.0000

Data Index: 103772 (The Pile)

Max Activating Token Index: 580

Text #3

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.9425. Min Act: -0.0000

Data Index: 1407247 (The Pile)

Max Activating Token Index: 24

Text #4

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8131. Min Act: -0.0000

Data Index: 1479365 (The Pile)

Max Activating Token Index: 164

Text #5

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8646. Min Act: -0.0000

Data Index: 261449 (The Pile)

Max Activating Token Index: 311

Text #6

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8484. Min Act: -0.0000

Data Index: 1080221 (The Pile)

Max Activating Token Index: 895

Text #7

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8392. Min Act: -0.0000

Data Index: 76336 (The Pile)

Max Activating Token Index: 327

Text #8

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8835. Min Act: -0.0000

Data Index: 1474450 (The Pile)

Max Activating Token Index: 980

Text #9

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8221. Min Act: -0.0000

Data Index: 901010 (The Pile)

Max Activating Token Index: 552

Text #10

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7696. Min Act: -0.0000

Data Index: 311532 (The Pile)

Max Activating Token Index: 1004

Text #11

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7499. Min Act: -0.0000

Data Index: 1086212 (The Pile)

Max Activating Token Index: 800

Text #12

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.6648. Min Act: -0.0000

Data Index: 816090 (The Pile)

Max Activating Token Index: 994

Text #13

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8007. Min Act: -0.0000

Data Index: 794612 (The Pile)

Max Activating Token Index: 91

Text #14

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7650. Min Act: -0.0000

Data Index: 1897123 (The Pile)

Max Activating Token Index: 142

Text #15

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.8105. Min Act: -0.0000

Data Index: 1349565 (The Pile)

Max Activating Token Index: 849

Text #16

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7782. Min Act: -0.0000

Data Index: 151337 (The Pile)

Max Activating Token Index: 1003

Text #17

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7744. Min Act: -0.0000

Data Index: 1402063 (The Pile)

Max Activating Token Index: 431

Text #18

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7703. Min Act: -0.0000

Data Index: 1246704 (The Pile)

Max Activating Token Index: 257

Text #19

Max Range: 1.0459. Min Range: -1.0459

Max Act: 0.7727. Min Act: -0.0000

Data Index: 1866162 (The Pile)

Max Activating Token Index: 824