Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3827 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.8117. Min Act: -0.0000

Data Index: 725354 (The Pile)

Max Activating Token Index: 781

Text #1

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.7783. Min Act: -0.0000

Data Index: 1080708 (The Pile)

Max Activating Token Index: 125

Text #2

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.7337. Min Act: -0.0000

Data Index: 422964 (The Pile)

Max Activating Token Index: 965

Text #3

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6771. Min Act: -0.0000

Data Index: 1129233 (The Pile)

Max Activating Token Index: 163

Text #4

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.5837. Min Act: -0.0000

Data Index: 822888 (The Pile)

Max Activating Token Index: 993

Text #5

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6396. Min Act: -0.0000

Data Index: 1452652 (The Pile)

Max Activating Token Index: 147

Text #6

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6450. Min Act: -0.0000

Data Index: 498561 (The Pile)

Max Activating Token Index: 477

Text #7

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.7252. Min Act: -0.0000

Data Index: 1575324 (The Pile)

Max Activating Token Index: 353

Text #8

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6199. Min Act: -0.0000

Data Index: 1904548 (The Pile)

Max Activating Token Index: 968

Text #9

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6439. Min Act: -0.0000

Data Index: 1862189 (The Pile)

Max Activating Token Index: 40

Text #10

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6592. Min Act: -0.0000

Data Index: 1479414 (The Pile)

Max Activating Token Index: 464

Text #11

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6551. Min Act: -0.0000

Data Index: 1666587 (The Pile)

Max Activating Token Index: 350

Text #12

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6280. Min Act: -0.0000

Data Index: 1815980 (The Pile)

Max Activating Token Index: 550

Text #13

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6235. Min Act: -0.0000

Data Index: 1485720 (The Pile)

Max Activating Token Index: 422

Text #14

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6161. Min Act: -0.0000

Data Index: 83223 (The Pile)

Max Activating Token Index: 371

Text #15

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.5957. Min Act: -0.0000

Data Index: 88005 (The Pile)

Max Activating Token Index: 465

Text #16

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6408. Min Act: -0.0000

Data Index: 1811414 (The Pile)

Max Activating Token Index: 758

Text #17

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.6920. Min Act: -0.0000

Data Index: 304746 (The Pile)

Max Activating Token Index: 376

Text #18

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.5943. Min Act: -0.0000

Data Index: 185612 (The Pile)

Max Activating Token Index: 735

Text #19

Max Range: 0.8117. Min Range: -0.8117

Max Act: 0.5865. Min Act: -0.0000

Data Index: 1881196 (The Pile)

Max Activating Token Index: 712