Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2068 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.5302. Min Act: -0.1700

Data Index: 8054683 (Open Web Text)

Max Activating Token Index: 378

Text #1

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3771. Min Act: -0.1700

Data Index: 7490992 (Open Web Text)

Max Activating Token Index: 489

Text #2

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.4067. Min Act: -0.1700

Data Index: 2562750 (Open Web Text)

Max Activating Token Index: 429

Text #3

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3714. Min Act: -0.1700

Data Index: 7049792 (Open Web Text)

Max Activating Token Index: 314

Text #4

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3895. Min Act: -0.1700

Data Index: 6925137 (Open Web Text)

Max Activating Token Index: 512

Text #5

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3492. Min Act: -0.1700

Data Index: 8256715 (Open Web Text)

Max Activating Token Index: 890

Text #6

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3368. Min Act: -0.1700

Data Index: 8266401 (Open Web Text)

Max Activating Token Index: 891

Text #7

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2834. Min Act: -0.1700

Data Index: 816669 (Open Web Text)

Max Activating Token Index: 342

Text #8

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.3196. Min Act: -0.1700

Data Index: 2618172 (Open Web Text)

Max Activating Token Index: 1017

Text #9

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2547. Min Act: -0.1700

Data Index: 8381793 (Open Web Text)

Max Activating Token Index: 109

Text #10

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2799. Min Act: -0.1700

Data Index: 1597497 (Open Web Text)

Max Activating Token Index: 686

Text #11

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2309. Min Act: -0.1700

Data Index: 8060305 (Open Web Text)

Max Activating Token Index: 568

Text #12

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2198. Min Act: -0.1700

Data Index: 4164144 (Open Web Text)

Max Activating Token Index: 1001

Text #13

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2261. Min Act: -0.1700

Data Index: 6370044 (Open Web Text)

Max Activating Token Index: 487

Text #14

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.1801. Min Act: -0.1700

Data Index: 3599704 (Open Web Text)

Max Activating Token Index: 334

Text #15

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.1837. Min Act: -0.1700

Data Index: 5824340 (Open Web Text)

Max Activating Token Index: 108

Text #16

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.1864. Min Act: -0.1700

Data Index: 4728099 (Open Web Text)

Max Activating Token Index: 971

Text #17

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.2088. Min Act: -0.1700

Data Index: 5340640 (Open Web Text)

Max Activating Token Index: 867

Text #18

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.1987. Min Act: -0.1700

Data Index: 874324 (Open Web Text)

Max Activating Token Index: 883

Text #19

Max Range: 4.5302. Min Range: -4.5302

Max Act: 4.1694. Min Act: -0.1700

Data Index: 2249030 (Open Web Text)

Max Activating Token Index: 133