Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 580 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.2982. Min Act: -0.0000

Data Index: 278505 (The Pile)

Max Activating Token Index: 910

Text #1

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.2457. Min Act: -0.0000

Data Index: 1753536 (The Pile)

Max Activating Token Index: 267

Text #2

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.0350. Min Act: -0.0000

Data Index: 433534 (The Pile)

Max Activating Token Index: 428

Text #3

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.0480. Min Act: -0.0000

Data Index: 1460196 (The Pile)

Max Activating Token Index: 176

Text #4

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.0094. Min Act: -0.0000

Data Index: 1224804 (The Pile)

Max Activating Token Index: 425

Text #5

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.0292. Min Act: -0.0000

Data Index: 417277 (The Pile)

Max Activating Token Index: 498

Text #6

Max Range: 1.2982. Min Range: -1.2982

Max Act: 1.0086. Min Act: -0.0000

Data Index: 283748 (The Pile)

Max Activating Token Index: 604

Text #7

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.9539. Min Act: -0.0000

Data Index: 1248687 (The Pile)

Max Activating Token Index: 37

Text #8

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8991. Min Act: -0.0000

Data Index: 661458 (The Pile)

Max Activating Token Index: 935

Text #9

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.9049. Min Act: -0.0000

Data Index: 1531394 (The Pile)

Max Activating Token Index: 595

Text #10

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8948. Min Act: -0.0000

Data Index: 1301985 (The Pile)

Max Activating Token Index: 645

Text #11

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.9226. Min Act: -0.0000

Data Index: 611427 (The Pile)

Max Activating Token Index: 991

Text #12

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.9091. Min Act: -0.0000

Data Index: 215889 (The Pile)

Max Activating Token Index: 942

Text #13

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8840. Min Act: -0.0000

Data Index: 389828 (The Pile)

Max Activating Token Index: 339

Text #14

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8774. Min Act: -0.0000

Data Index: 983709 (The Pile)

Max Activating Token Index: 439

Text #15

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8498. Min Act: -0.0000

Data Index: 1309164 (The Pile)

Max Activating Token Index: 652

Text #16

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8699. Min Act: -0.0000

Data Index: 658140 (The Pile)

Max Activating Token Index: 283

Text #17

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8259. Min Act: -0.0000

Data Index: 181513 (The Pile)

Max Activating Token Index: 154

Text #18

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8054. Min Act: -0.0000

Data Index: 180976 (The Pile)

Max Activating Token Index: 821

Text #19

Max Range: 1.2982. Min Range: -1.2982

Max Act: 0.8242. Min Act: -0.0000

Data Index: 523368 (The Pile)

Max Activating Token Index: 862