Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 813 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.3196. Min Act: -0.1700

Data Index: 6541571 (Open Web Text)

Max Activating Token Index: 33

Text #1

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2823. Min Act: -0.1700

Data Index: 2937117 (Open Web Text)

Max Activating Token Index: 510

Text #2

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.3129. Min Act: -0.1700

Data Index: 6508198 (Open Web Text)

Max Activating Token Index: 588

Text #3

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.3005. Min Act: -0.1700

Data Index: 3398468 (Open Web Text)

Max Activating Token Index: 109

Text #4

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2901. Min Act: -0.1700

Data Index: 3693492 (Open Web Text)

Max Activating Token Index: 86

Text #5

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2694. Min Act: -0.1700

Data Index: 7550182 (Open Web Text)

Max Activating Token Index: 341

Text #6

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2646. Min Act: -0.1700

Data Index: 7402477 (Open Web Text)

Max Activating Token Index: 380

Text #7

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2648. Min Act: -0.1700

Data Index: 5140402 (Open Web Text)

Max Activating Token Index: 420

Text #8

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2435. Min Act: -0.1700

Data Index: 7311471 (Open Web Text)

Max Activating Token Index: 35

Text #9

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2563. Min Act: -0.1700

Data Index: 752100 (Open Web Text)

Max Activating Token Index: 157

Text #10

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2360. Min Act: -0.1700

Data Index: 7610537 (Open Web Text)

Max Activating Token Index: 205

Text #11

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2582. Min Act: -0.1700

Data Index: 1339830 (Open Web Text)

Max Activating Token Index: 552

Text #12

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2656. Min Act: -0.1700

Data Index: 7158500 (Open Web Text)

Max Activating Token Index: 899

Text #13

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2814. Min Act: -0.1700

Data Index: 3464119 (Open Web Text)

Max Activating Token Index: 399

Text #14

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2250. Min Act: -0.1700

Data Index: 3988349 (Open Web Text)

Max Activating Token Index: 67

Text #15

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2267. Min Act: -0.1700

Data Index: 1535151 (Open Web Text)

Max Activating Token Index: 202

Text #16

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2255. Min Act: -0.1700

Data Index: 4832350 (Open Web Text)

Max Activating Token Index: 874

Text #17

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2266. Min Act: -0.1700

Data Index: 6872948 (Open Web Text)

Max Activating Token Index: 139

Text #18

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.2291. Min Act: -0.1700

Data Index: 8168562 (Open Web Text)

Max Activating Token Index: 88

Text #19

Max Range: 3.3196. Min Range: -3.3196

Max Act: 3.1970. Min Act: -0.1700

Data Index: 180230 (Open Web Text)

Max Activating Token Index: 446