Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2779 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.3665. Min Act: -0.0000

Data Index: 1166435 (The Pile)

Max Activating Token Index: 876

Text #1

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.5486. Min Act: -0.0000

Data Index: 340690 (The Pile)

Max Activating Token Index: 807

Text #2

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.5981. Min Act: -0.0000

Data Index: 32855 (The Pile)

Max Activating Token Index: 568

Text #3

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.2765. Min Act: -0.0000

Data Index: 776150 (The Pile)

Max Activating Token Index: 568

Text #4

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.1924. Min Act: -0.0000

Data Index: 1300023 (The Pile)

Max Activating Token Index: 833

Text #5

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0805. Min Act: -0.0000

Data Index: 1624189 (The Pile)

Max Activating Token Index: 88

Text #6

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0037. Min Act: 0.0000

Data Index: 768281 (The Pile)

Max Activating Token Index: 794

Text #7

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0341. Min Act: -0.0000

Data Index: 1026269 (The Pile)

Max Activating Token Index: 198

Text #8

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9781. Min Act: -0.0000

Data Index: 990102 (The Pile)

Max Activating Token Index: 1010

Text #9

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0443. Min Act: -0.0000

Data Index: 1367288 (The Pile)

Max Activating Token Index: 469

Text #10

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0390. Min Act: -0.0000

Data Index: 1188711 (The Pile)

Max Activating Token Index: 898

Text #11

Max Range: 1.5981. Min Range: -1.5981

Max Act: 1.0382. Min Act: -0.0000

Data Index: 456683 (The Pile)

Max Activating Token Index: 458

Text #12

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9299. Min Act: -0.0000

Data Index: 710745 (The Pile)

Max Activating Token Index: 986

Text #13

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9420. Min Act: -0.0000

Data Index: 1640736 (The Pile)

Max Activating Token Index: 938

Text #14

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9485. Min Act: -0.0000

Data Index: 641546 (The Pile)

Max Activating Token Index: 971

Text #15

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9191. Min Act: -0.0000

Data Index: 409914 (The Pile)

Max Activating Token Index: 506

Text #16

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.8991. Min Act: -0.0000

Data Index: 309465 (The Pile)

Max Activating Token Index: 323

Text #17

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9287. Min Act: -0.0000

Data Index: 963612 (The Pile)

Max Activating Token Index: 1023

Text #18

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.9478. Min Act: -0.0000

Data Index: 256603 (The Pile)

Max Activating Token Index: 26

Text #19

Max Range: 1.5981. Min Range: -1.5981

Max Act: 0.8662. Min Act: -0.0000

Data Index: 1600320 (The Pile)

Max Activating Token Index: 724