Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2289 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.6203. Min Act: -0.1700

Data Index: 4848281 (Open Web Text)

Max Activating Token Index: 421

Text #1

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.5879. Min Act: -0.1700

Data Index: 2904926 (Open Web Text)

Max Activating Token Index: 383

Text #2

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.5808. Min Act: -0.1700

Data Index: 2904925 (Open Web Text)

Max Activating Token Index: 823

Text #3

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.4004. Min Act: -0.1700

Data Index: 8811158 (Open Web Text)

Max Activating Token Index: 718

Text #4

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.4770. Min Act: -0.1700

Data Index: 4197852 (Open Web Text)

Max Activating Token Index: 208

Text #5

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.4267. Min Act: -0.1700

Data Index: 8011689 (Open Web Text)

Max Activating Token Index: 228

Text #6

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.4490. Min Act: -0.1700

Data Index: 5015439 (Open Web Text)

Max Activating Token Index: 484

Text #7

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3915. Min Act: -0.1700

Data Index: 7014503 (Open Web Text)

Max Activating Token Index: 209

Text #8

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3859. Min Act: -0.1700

Data Index: 6700944 (Open Web Text)

Max Activating Token Index: 941

Text #9

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3797. Min Act: -0.1700

Data Index: 2937824 (Open Web Text)

Max Activating Token Index: 788

Text #10

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3543. Min Act: -0.1700

Data Index: 8655290 (Open Web Text)

Max Activating Token Index: 244

Text #11

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3456. Min Act: -0.1700

Data Index: 8023009 (Open Web Text)

Max Activating Token Index: 280

Text #12

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3466. Min Act: -0.1700

Data Index: 5806991 (Open Web Text)

Max Activating Token Index: 64

Text #13

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3108. Min Act: -0.1700

Data Index: 6636131 (Open Web Text)

Max Activating Token Index: 63

Text #14

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3610. Min Act: -0.1700

Data Index: 2268946 (Open Web Text)

Max Activating Token Index: 898

Text #15

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3145. Min Act: -0.1700

Data Index: 3224766 (Open Web Text)

Max Activating Token Index: 288

Text #16

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.2852. Min Act: -0.1700

Data Index: 1135777 (Open Web Text)

Max Activating Token Index: 991

Text #17

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.2888. Min Act: -0.1700

Data Index: 6883608 (Open Web Text)

Max Activating Token Index: 683

Text #18

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.3003. Min Act: -0.1700

Data Index: 2079165 (Open Web Text)

Max Activating Token Index: 184

Text #19

Max Range: 4.6203. Min Range: -4.6203

Max Act: 4.2849. Min Act: -0.1700

Data Index: 6848706 (Open Web Text)

Max Activating Token Index: 63