Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1772 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.3022. Min Act: -0.1700

Data Index: 5009028 (Open Web Text)

Max Activating Token Index: 750

Text #1

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2625. Min Act: -0.1700

Data Index: 8228315 (Open Web Text)

Max Activating Token Index: 936

Text #2

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2869. Min Act: -0.1700

Data Index: 4465131 (Open Web Text)

Max Activating Token Index: 295

Text #3

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2348. Min Act: -0.1700

Data Index: 8609871 (Open Web Text)

Max Activating Token Index: 452

Text #4

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2221. Min Act: -0.1700

Data Index: 2321715 (Open Web Text)

Max Activating Token Index: 213

Text #5

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2113. Min Act: -0.1700

Data Index: 6802640 (Open Web Text)

Max Activating Token Index: 767

Text #6

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1861. Min Act: -0.1700

Data Index: 5071007 (Open Web Text)

Max Activating Token Index: 588

Text #7

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.2081. Min Act: -0.1700

Data Index: 3438561 (Open Web Text)

Max Activating Token Index: 261

Text #8

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1687. Min Act: -0.1700

Data Index: 7573337 (Open Web Text)

Max Activating Token Index: 216

Text #9

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1701. Min Act: -0.1700

Data Index: 5890185 (Open Web Text)

Max Activating Token Index: 252

Text #10

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1310. Min Act: -0.1700

Data Index: 1194880 (Open Web Text)

Max Activating Token Index: 340

Text #11

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1315. Min Act: -0.1700

Data Index: 6384563 (Open Web Text)

Max Activating Token Index: 216

Text #12

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1341. Min Act: -0.1700

Data Index: 5204985 (Open Web Text)

Max Activating Token Index: 124

Text #13

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1367. Min Act: -0.1700

Data Index: 5419186 (Open Web Text)

Max Activating Token Index: 961

Text #14

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1125. Min Act: -0.1700

Data Index: 2563449 (Open Web Text)

Max Activating Token Index: 942

Text #15

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1076. Min Act: -0.1700

Data Index: 2781676 (Open Web Text)

Max Activating Token Index: 865

Text #16

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.1320. Min Act: -0.1700

Data Index: 1273315 (Open Web Text)

Max Activating Token Index: 166

Text #17

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.0395. Min Act: -0.1700

Data Index: 2370351 (Open Web Text)

Max Activating Token Index: 490

Text #18

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.0711. Min Act: -0.1700

Data Index: 2613812 (Open Web Text)

Max Activating Token Index: 56

Text #19

Max Range: 4.3022. Min Range: -4.3022

Max Act: 4.0692. Min Act: -0.1700

Data Index: 273131 (Open Web Text)

Max Activating Token Index: 62