Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 623 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.5648. Min Act: -0.1700

Data Index: 7365880 (Open Web Text)

Max Activating Token Index: 879

Text #1

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4985. Min Act: -0.1700

Data Index: 591939 (Open Web Text)

Max Activating Token Index: 877

Text #2

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4914. Min Act: -0.1700

Data Index: 36815 (Open Web Text)

Max Activating Token Index: 163

Text #3

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4815. Min Act: -0.1700

Data Index: 767453 (Open Web Text)

Max Activating Token Index: 964

Text #4

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4646. Min Act: -0.1700

Data Index: 1761804 (Open Web Text)

Max Activating Token Index: 1007

Text #5

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4419. Min Act: -0.1700

Data Index: 3854127 (Open Web Text)

Max Activating Token Index: 921

Text #6

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4050. Min Act: -0.1700

Data Index: 3969578 (Open Web Text)

Max Activating Token Index: 518

Text #7

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4154. Min Act: -0.1700

Data Index: 6170558 (Open Web Text)

Max Activating Token Index: 198

Text #8

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4070. Min Act: -0.1700

Data Index: 1135269 (Open Web Text)

Max Activating Token Index: 640

Text #9

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4473. Min Act: -0.1700

Data Index: 3271553 (Open Web Text)

Max Activating Token Index: 918

Text #10

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3808. Min Act: -0.1700

Data Index: 2336212 (Open Web Text)

Max Activating Token Index: 717

Text #11

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3368. Min Act: -0.1700

Data Index: 4531694 (Open Web Text)

Max Activating Token Index: 370

Text #12

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4070. Min Act: -0.1700

Data Index: 2040433 (Open Web Text)

Max Activating Token Index: 471

Text #13

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.4045. Min Act: -0.1700

Data Index: 1006492 (Open Web Text)

Max Activating Token Index: 982

Text #14

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3662. Min Act: -0.1700

Data Index: 1483238 (Open Web Text)

Max Activating Token Index: 991

Text #15

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.2993. Min Act: -0.1700

Data Index: 3505280 (Open Web Text)

Max Activating Token Index: 943

Text #16

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3414. Min Act: -0.1700

Data Index: 5097430 (Open Web Text)

Max Activating Token Index: 668

Text #17

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3303. Min Act: -0.1700

Data Index: 5104293 (Open Web Text)

Max Activating Token Index: 412

Text #18

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3648. Min Act: -0.1700

Data Index: 5140021 (Open Web Text)

Max Activating Token Index: 17

Text #19

Max Range: 4.5648. Min Range: -4.5648

Max Act: 4.3430. Min Act: -0.1700

Data Index: 6205877 (Open Web Text)

Max Activating Token Index: 1016