Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 379 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.5290. Min Act: -0.1700

Data Index: 6617712 (Open Web Text)

Max Activating Token Index: 72

Text #1

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3856. Min Act: -0.1700

Data Index: 5398147 (Open Web Text)

Max Activating Token Index: 415

Text #2

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3447. Min Act: -0.1700

Data Index: 6399031 (Open Web Text)

Max Activating Token Index: 107

Text #3

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3665. Min Act: -0.1700

Data Index: 1850501 (Open Web Text)

Max Activating Token Index: 418

Text #4

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3372. Min Act: -0.1700

Data Index: 1956864 (Open Web Text)

Max Activating Token Index: 238

Text #5

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2809. Min Act: -0.1700

Data Index: 8118679 (Open Web Text)

Max Activating Token Index: 971

Text #6

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2982. Min Act: -0.1700

Data Index: 2308936 (Open Web Text)

Max Activating Token Index: 17

Text #7

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2812. Min Act: -0.1700

Data Index: 1476544 (Open Web Text)

Max Activating Token Index: 81

Text #8

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3099. Min Act: -0.1700

Data Index: 3312847 (Open Web Text)

Max Activating Token Index: 728

Text #9

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2867. Min Act: -0.1700

Data Index: 3346089 (Open Web Text)

Max Activating Token Index: 465

Text #10

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.3075. Min Act: -0.1700

Data Index: 3955049 (Open Web Text)

Max Activating Token Index: 290

Text #11

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2852. Min Act: -0.1700

Data Index: 70611 (Open Web Text)

Max Activating Token Index: 920

Text #12

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2517. Min Act: -0.1700

Data Index: 1445350 (Open Web Text)

Max Activating Token Index: 326

Text #13

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2367. Min Act: -0.1700

Data Index: 4348101 (Open Web Text)

Max Activating Token Index: 812

Text #14

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2778. Min Act: -0.1700

Data Index: 3123514 (Open Web Text)

Max Activating Token Index: 502

Text #15

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2542. Min Act: -0.1700

Data Index: 4546042 (Open Web Text)

Max Activating Token Index: 139

Text #16

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2745. Min Act: -0.1700

Data Index: 5491696 (Open Web Text)

Max Activating Token Index: 985

Text #17

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2646. Min Act: -0.1700

Data Index: 6260891 (Open Web Text)

Max Activating Token Index: 396

Text #18

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2692. Min Act: -0.1700

Data Index: 1305326 (Open Web Text)

Max Activating Token Index: 936

Text #19

Max Range: 4.5290. Min Range: -4.5290

Max Act: 4.2719. Min Act: -0.1700

Data Index: 2695398 (Open Web Text)

Max Activating Token Index: 380