Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2824 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.8620. Min Act: -0.1700

Data Index: 4467622 (Open Web Text)

Max Activating Token Index: 910

Text #1

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.8253. Min Act: -0.1700

Data Index: 3000695 (Open Web Text)

Max Activating Token Index: 305

Text #2

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.8088. Min Act: -0.1700

Data Index: 7562201 (Open Web Text)

Max Activating Token Index: 605

Text #3

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7973. Min Act: -0.1700

Data Index: 551836 (Open Web Text)

Max Activating Token Index: 920

Text #4

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7163. Min Act: -0.1700

Data Index: 6349524 (Open Web Text)

Max Activating Token Index: 670

Text #5

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7649. Min Act: -0.1700

Data Index: 3571087 (Open Web Text)

Max Activating Token Index: 496

Text #6

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7681. Min Act: -0.1700

Data Index: 5696331 (Open Web Text)

Max Activating Token Index: 229

Text #7

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7138. Min Act: -0.1700

Data Index: 5262102 (Open Web Text)

Max Activating Token Index: 73

Text #8

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7064. Min Act: -0.1700

Data Index: 3076966 (Open Web Text)

Max Activating Token Index: 241

Text #9

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7088. Min Act: -0.1700

Data Index: 5356019 (Open Web Text)

Max Activating Token Index: 641

Text #10

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7081. Min Act: -0.1700

Data Index: 2211792 (Open Web Text)

Max Activating Token Index: 552

Text #11

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.7180. Min Act: -0.1700

Data Index: 372150 (Open Web Text)

Max Activating Token Index: 488

Text #12

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6670. Min Act: -0.1700

Data Index: 5398326 (Open Web Text)

Max Activating Token Index: 98

Text #13

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6995. Min Act: -0.1700

Data Index: 1583085 (Open Web Text)

Max Activating Token Index: 72

Text #14

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6628. Min Act: -0.1700

Data Index: 8047546 (Open Web Text)

Max Activating Token Index: 751

Text #15

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6935. Min Act: -0.1700

Data Index: 2531440 (Open Web Text)

Max Activating Token Index: 926

Text #16

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6877. Min Act: -0.1700

Data Index: 2815178 (Open Web Text)

Max Activating Token Index: 819

Text #17

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6426. Min Act: -0.1700

Data Index: 6815495 (Open Web Text)

Max Activating Token Index: 751

Text #18

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6603. Min Act: -0.1700

Data Index: 113092 (Open Web Text)

Max Activating Token Index: 498

Text #19

Max Range: 3.8620. Min Range: -3.8620

Max Act: 3.6624. Min Act: -0.1700

Data Index: 8632496 (Open Web Text)

Max Activating Token Index: 903