Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 374 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.9253. Min Act: -0.1700

Data Index: 5104608 (Open Web Text)

Max Activating Token Index: 76

Text #1

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8484. Min Act: -0.1700

Data Index: 8432518 (Open Web Text)

Max Activating Token Index: 543

Text #2

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8493. Min Act: -0.1700

Data Index: 2497208 (Open Web Text)

Max Activating Token Index: 400

Text #3

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8536. Min Act: -0.1700

Data Index: 7292935 (Open Web Text)

Max Activating Token Index: 313

Text #4

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8417. Min Act: -0.1700

Data Index: 1492640 (Open Web Text)

Max Activating Token Index: 932

Text #5

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8370. Min Act: -0.1700

Data Index: 2122498 (Open Web Text)

Max Activating Token Index: 329

Text #6

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8373. Min Act: -0.1700

Data Index: 3305722 (Open Web Text)

Max Activating Token Index: 400

Text #7

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8330. Min Act: -0.1700

Data Index: 7624594 (Open Web Text)

Max Activating Token Index: 566

Text #8

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8206. Min Act: -0.1700

Data Index: 7115267 (Open Web Text)

Max Activating Token Index: 107

Text #9

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8163. Min Act: -0.1700

Data Index: 4420427 (Open Web Text)

Max Activating Token Index: 660

Text #10

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8289. Min Act: -0.1700

Data Index: 8782898 (Open Web Text)

Max Activating Token Index: 452

Text #11

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8186. Min Act: -0.1700

Data Index: 8016467 (Open Web Text)

Max Activating Token Index: 949

Text #12

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8181. Min Act: -0.1700

Data Index: 7722021 (Open Web Text)

Max Activating Token Index: 54

Text #13

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8032. Min Act: -0.1699

Data Index: 5637916 (Open Web Text)

Max Activating Token Index: 191

Text #14

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8075. Min Act: -0.1700

Data Index: 1652159 (Open Web Text)

Max Activating Token Index: 646

Text #15

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8049. Min Act: -0.1700

Data Index: 7237984 (Open Web Text)

Max Activating Token Index: 301

Text #16

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8150. Min Act: -0.1700

Data Index: 7229404 (Open Web Text)

Max Activating Token Index: 985

Text #17

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8050. Min Act: -0.1700

Data Index: 5512654 (Open Web Text)

Max Activating Token Index: 661

Text #18

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8113. Min Act: -0.1700

Data Index: 3110195 (Open Web Text)

Max Activating Token Index: 284

Text #19

Max Range: 1.9253. Min Range: -1.9253

Max Act: 1.8065. Min Act: -0.1700

Data Index: 729271 (Open Web Text)

Max Activating Token Index: 448