Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 925 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.1364. Min Act: -0.0000

Data Index: 858648 (The Pile)

Max Activating Token Index: 354

Text #1

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.0812. Min Act: -0.0000

Data Index: 946399 (The Pile)

Max Activating Token Index: 382

Text #2

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.2774. Min Act: -0.0000

Data Index: 1431766 (The Pile)

Max Activating Token Index: 840

Text #3

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.1384. Min Act: -0.0000

Data Index: 1639705 (The Pile)

Max Activating Token Index: 648

Text #4

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.9658. Min Act: -0.0000

Data Index: 1378189 (The Pile)

Max Activating Token Index: 524

Text #5

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.1999. Min Act: -0.0000

Data Index: 212530 (The Pile)

Max Activating Token Index: 956

Text #6

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.0712. Min Act: -0.0000

Data Index: 1463788 (The Pile)

Max Activating Token Index: 85

Text #7

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.9716. Min Act: -0.0000

Data Index: 205916 (The Pile)

Max Activating Token Index: 717

Text #8

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.1145. Min Act: -0.0000

Data Index: 1955381 (The Pile)

Max Activating Token Index: 338

Text #9

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.7314. Min Act: -0.0000

Data Index: 114099 (The Pile)

Max Activating Token Index: 1020

Text #10

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.6854. Min Act: -0.0000

Data Index: 81867 (The Pile)

Max Activating Token Index: 805

Text #11

Max Range: 5.2774. Min Range: -5.2774

Max Act: 5.0617. Min Act: -0.0000

Data Index: 90632 (The Pile)

Max Activating Token Index: 467

Text #12

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.8532. Min Act: -0.0000

Data Index: 18522 (The Pile)

Max Activating Token Index: 380

Text #13

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.6137. Min Act: -0.0000

Data Index: 1724979 (The Pile)

Max Activating Token Index: 241

Text #14

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.7444. Min Act: -0.0000

Data Index: 811423 (The Pile)

Max Activating Token Index: 61

Text #15

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.7859. Min Act: -0.0000

Data Index: 382608 (The Pile)

Max Activating Token Index: 156

Text #16

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.9385. Min Act: -0.0000

Data Index: 1174993 (The Pile)

Max Activating Token Index: 336

Text #17

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.9047. Min Act: -0.0000

Data Index: 1710290 (The Pile)

Max Activating Token Index: 176

Text #18

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.5343. Min Act: -0.0000

Data Index: 1208617 (The Pile)

Max Activating Token Index: 430

Text #19

Max Range: 5.2774. Min Range: -5.2774

Max Act: 4.7122. Min Act: -0.0000

Data Index: 495561 (The Pile)

Max Activating Token Index: 522