Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1400 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.1138. Min Range: -1.1138

Max Act: 1.0917. Min Act: -0.0000

Data Index: 1475579 (The Pile)

Max Activating Token Index: 592

Text #1

Max Range: 1.1138. Min Range: -1.1138

Max Act: 1.0466. Min Act: 0.0000

Data Index: 1620515 (The Pile)

Max Activating Token Index: 527

Text #2

Max Range: 1.1138. Min Range: -1.1138

Max Act: 1.1138. Min Act: 0.0000

Data Index: 1456930 (The Pile)

Max Activating Token Index: 758

Text #3

Max Range: 1.1138. Min Range: -1.1138

Max Act: 1.0348. Min Act: -0.0000

Data Index: 1417326 (The Pile)

Max Activating Token Index: 509

Text #4

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.9472. Min Act: -0.0000

Data Index: 1934179 (The Pile)

Max Activating Token Index: 83

Text #5

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.9423. Min Act: -0.0000

Data Index: 1493676 (The Pile)

Max Activating Token Index: 859

Text #6

Max Range: 1.1138. Min Range: -1.1138

Max Act: 1.0307. Min Act: -0.0000

Data Index: 1369737 (The Pile)

Max Activating Token Index: 143

Text #7

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.9253. Min Act: -0.0000

Data Index: 1013615 (The Pile)

Max Activating Token Index: 428

Text #8

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.9514. Min Act: -0.0000

Data Index: 224759 (The Pile)

Max Activating Token Index: 277

Text #9

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8952. Min Act: -0.0000

Data Index: 646328 (The Pile)

Max Activating Token Index: 409

Text #10

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8564. Min Act: -0.0000

Data Index: 1702577 (The Pile)

Max Activating Token Index: 759

Text #11

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8633. Min Act: -0.0000

Data Index: 1732823 (The Pile)

Max Activating Token Index: 217

Text #12

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8429. Min Act: -0.0000

Data Index: 1427611 (The Pile)

Max Activating Token Index: 819

Text #13

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8541. Min Act: -0.0000

Data Index: 89506 (The Pile)

Max Activating Token Index: 292

Text #14

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8428. Min Act: -0.0000

Data Index: 821091 (The Pile)

Max Activating Token Index: 603

Text #15

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8362. Min Act: -0.0000

Data Index: 1550538 (The Pile)

Max Activating Token Index: 320

Text #16

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8574. Min Act: -0.0000

Data Index: 456877 (The Pile)

Max Activating Token Index: 960

Text #17

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.7701. Min Act: -0.0000

Data Index: 980242 (The Pile)

Max Activating Token Index: 44

Text #18

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.7926. Min Act: -0.0000

Data Index: 484967 (The Pile)

Max Activating Token Index: 642

Text #19

Max Range: 1.1138. Min Range: -1.1138

Max Act: 0.8951. Min Act: -0.0000

Data Index: 1941936 (The Pile)

Max Activating Token Index: 158