Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1569 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.2313. Min Act: -0.1700

Data Index: 4819323 (Open Web Text)

Max Activating Token Index: 528

Text #1

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.2055. Min Act: -0.1700

Data Index: 2208589 (Open Web Text)

Max Activating Token Index: 817

Text #2

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1639. Min Act: -0.1700

Data Index: 1594723 (Open Web Text)

Max Activating Token Index: 891

Text #3

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1454. Min Act: -0.1700

Data Index: 5277460 (Open Web Text)

Max Activating Token Index: 1017

Text #4

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1466. Min Act: -0.1700

Data Index: 602931 (Open Web Text)

Max Activating Token Index: 285

Text #5

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1468. Min Act: -0.1700

Data Index: 8599894 (Open Web Text)

Max Activating Token Index: 521

Text #6

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1412. Min Act: -0.1700

Data Index: 8566107 (Open Web Text)

Max Activating Token Index: 17

Text #7

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1493. Min Act: -0.1700

Data Index: 894864 (Open Web Text)

Max Activating Token Index: 105

Text #8

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1298. Min Act: -0.1700

Data Index: 1040345 (Open Web Text)

Max Activating Token Index: 927

Text #9

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1333. Min Act: -0.1700

Data Index: 3963354 (Open Web Text)

Max Activating Token Index: 396

Text #10

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1150. Min Act: -0.1700

Data Index: 1594724 (Open Web Text)

Max Activating Token Index: 154

Text #11

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1273. Min Act: -0.1700

Data Index: 3458660 (Open Web Text)

Max Activating Token Index: 201

Text #12

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1276. Min Act: -0.1700

Data Index: 7285956 (Open Web Text)

Max Activating Token Index: 926

Text #13

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.0952. Min Act: -0.1700

Data Index: 1058207 (Open Web Text)

Max Activating Token Index: 678

Text #14

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1113. Min Act: -0.1700

Data Index: 4186853 (Open Web Text)

Max Activating Token Index: 35

Text #15

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1216. Min Act: -0.1700

Data Index: 4186851 (Open Web Text)

Max Activating Token Index: 347

Text #16

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1167. Min Act: -0.1700

Data Index: 1259485 (Open Web Text)

Max Activating Token Index: 172

Text #17

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.1218. Min Act: -0.1700

Data Index: 130891 (Open Web Text)

Max Activating Token Index: 159

Text #18

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.0937. Min Act: -0.1700

Data Index: 2718168 (Open Web Text)

Max Activating Token Index: 100

Text #19

Max Range: 3.2313. Min Range: -3.2313

Max Act: 3.0992. Min Act: -0.1700

Data Index: 5212693 (Open Web Text)

Max Activating Token Index: 29