Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 2276 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l')

Text #0

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.3039. Min Act: -0.0000

Data Index: 118656 (C4 (Web Text))

Max Activating Token Index: 241

Text #1

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2805. Min Act: -0.0000

Data Index: 693639 (C4 (Web Text))

Max Activating Token Index: 522

Text #2

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2605. Min Act: -0.0000

Data Index: 660374 (C4 (Web Text))

Max Activating Token Index: 856

Text #3

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2611. Min Act: -0.0000

Data Index: 1195294 (C4 (Web Text))

Max Activating Token Index: 488

Text #4

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2662. Min Act: -0.0000

Data Index: 610688 (C4 (Web Text))

Max Activating Token Index: 832

Text #5

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2655. Min Act: -0.0000

Data Index: 1322179 (C4 (Web Text))

Max Activating Token Index: 113

Text #6

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2668. Min Act: -0.0000

Data Index: 813673 (C4 (Web Text))

Max Activating Token Index: 219

Text #7

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2487. Min Act: -0.0000

Data Index: 521262 (C4 (Web Text))

Max Activating Token Index: 34

Text #8

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2283. Min Act: -0.0000

Data Index: 1083273 (C4 (Web Text))

Max Activating Token Index: 198

Text #9

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2463. Min Act: -0.0001

Data Index: 690237 (C4 (Web Text))

Max Activating Token Index: 211

Text #10

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2505. Min Act: -0.0000

Data Index: 1171290 (C4 (Web Text))

Max Activating Token Index: 675

Text #11

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2405. Min Act: -0.0001

Data Index: 3556 (C4 (Web Text))

Max Activating Token Index: 445

Text #12

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2306. Min Act: -0.0000

Data Index: 507971 (C4 (Web Text))

Max Activating Token Index: 997

Text #13

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2325. Min Act: -0.0000

Data Index: 692049 (C4 (Web Text))

Max Activating Token Index: 35

Text #14

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2314. Min Act: -0.0000

Data Index: 402281 (C4 (Web Text))

Max Activating Token Index: 486

Text #15

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2315. Min Act: -0.0000

Data Index: 7096 (C4 (Web Text))

Max Activating Token Index: 559

Text #16

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2287. Min Act: -0.0000

Data Index: 1282649 (C4 (Web Text))

Max Activating Token Index: 978

Text #17

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2187. Min Act: -0.0000

Data Index: 505498 (C4 (Web Text))

Max Activating Token Index: 25

Text #18

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2185. Min Act: -0.0001

Data Index: 1101512 (C4 (Web Text))

Max Activating Token Index: 905

Text #19

Max Range: 0.3039. Min Range: -0.3039

Max Act: 0.2237. Min Act: -0.0000

Data Index: 656873 (C4 (Web Text))

Max Activating Token Index: 454