Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 474 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0263. Min Range: -3.0263

Max Act: 3.0263. Min Act: -0.1700

Data Index: 1033578 (Open Web Text)

Max Activating Token Index: 38

Text #1

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9753. Min Act: -0.1700

Data Index: 2856796 (Open Web Text)

Max Activating Token Index: 133

Text #2

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9601. Min Act: -0.1700

Data Index: 1033579 (Open Web Text)

Max Activating Token Index: 520

Text #3

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9711. Min Act: -0.1700

Data Index: 3722477 (Open Web Text)

Max Activating Token Index: 866

Text #4

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9383. Min Act: -0.1700

Data Index: 7369720 (Open Web Text)

Max Activating Token Index: 441

Text #5

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9419. Min Act: -0.1700

Data Index: 3914075 (Open Web Text)

Max Activating Token Index: 548

Text #6

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9213. Min Act: -0.1700

Data Index: 421138 (Open Web Text)

Max Activating Token Index: 266

Text #7

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9304. Min Act: -0.1700

Data Index: 5598192 (Open Web Text)

Max Activating Token Index: 745

Text #8

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9214. Min Act: -0.1700

Data Index: 2293123 (Open Web Text)

Max Activating Token Index: 384

Text #9

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9118. Min Act: -0.1700

Data Index: 4046854 (Open Web Text)

Max Activating Token Index: 769

Text #10

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9345. Min Act: -0.1700

Data Index: 5980561 (Open Web Text)

Max Activating Token Index: 543

Text #11

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9207. Min Act: -0.1700

Data Index: 279802 (Open Web Text)

Max Activating Token Index: 126

Text #12

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9437. Min Act: -0.1700

Data Index: 4118546 (Open Web Text)

Max Activating Token Index: 447

Text #13

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9242. Min Act: -0.1700

Data Index: 269901 (Open Web Text)

Max Activating Token Index: 202

Text #14

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9000. Min Act: -0.1700

Data Index: 2957048 (Open Web Text)

Max Activating Token Index: 362

Text #15

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.8959. Min Act: -0.1700

Data Index: 4861850 (Open Web Text)

Max Activating Token Index: 465

Text #16

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.8895. Min Act: -0.1700

Data Index: 770699 (Open Web Text)

Max Activating Token Index: 51

Text #17

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9147. Min Act: -0.1700

Data Index: 5468700 (Open Web Text)

Max Activating Token Index: 194

Text #18

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9050. Min Act: -0.1700

Data Index: 3354593 (Open Web Text)

Max Activating Token Index: 371

Text #19

Max Range: 3.0263. Min Range: -3.0263

Max Act: 2.9005. Min Act: -0.1700

Data Index: 1922732 (Open Web Text)

Max Activating Token Index: 925