Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 417 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.1066. Min Range: -1.1066

Max Act: 1.1066. Min Act: -0.0000

Data Index: 202481 (The Pile)

Max Activating Token Index: 824

Text #1

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.9598. Min Act: -0.0000

Data Index: 870264 (The Pile)

Max Activating Token Index: 60

Text #2

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.9420. Min Act: -0.0000

Data Index: 719186 (The Pile)

Max Activating Token Index: 792

Text #3

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.9309. Min Act: -0.0000

Data Index: 861938 (The Pile)

Max Activating Token Index: 939

Text #4

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.9915. Min Act: -0.0000

Data Index: 1534596 (The Pile)

Max Activating Token Index: 971

Text #5

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8627. Min Act: -0.0000

Data Index: 1393499 (The Pile)

Max Activating Token Index: 982

Text #6

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8434. Min Act: -0.0000

Data Index: 1672097 (The Pile)

Max Activating Token Index: 433

Text #7

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.9224. Min Act: -0.0000

Data Index: 1278802 (The Pile)

Max Activating Token Index: 151

Text #8

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8270. Min Act: -0.0000

Data Index: 1912101 (The Pile)

Max Activating Token Index: 380

Text #9

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8873. Min Act: -0.0000

Data Index: 1498933 (The Pile)

Max Activating Token Index: 511

Text #10

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7758. Min Act: -0.0000

Data Index: 169367 (The Pile)

Max Activating Token Index: 949

Text #11

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7626. Min Act: -0.0000

Data Index: 606497 (The Pile)

Max Activating Token Index: 412

Text #12

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8134. Min Act: -0.0000

Data Index: 1993056 (The Pile)

Max Activating Token Index: 453

Text #13

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7175. Min Act: -0.0000

Data Index: 1438422 (The Pile)

Max Activating Token Index: 1013

Text #14

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7856. Min Act: -0.0000

Data Index: 1594175 (The Pile)

Max Activating Token Index: 194

Text #15

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8363. Min Act: -0.0000

Data Index: 127366 (The Pile)

Max Activating Token Index: 955

Text #16

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7719. Min Act: -0.0000

Data Index: 939879 (The Pile)

Max Activating Token Index: 33

Text #17

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8008. Min Act: -0.0000

Data Index: 1299697 (The Pile)

Max Activating Token Index: 788

Text #18

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.8286. Min Act: -0.0000

Data Index: 511723 (The Pile)

Max Activating Token Index: 714

Text #19

Max Range: 1.1066. Min Range: -1.1066

Max Act: 0.7529. Min Act: -0.0000

Data Index: 32991 (The Pile)

Max Activating Token Index: 876