Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 999 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.5334. Min Act: -0.1700

Data Index: 6137672 (Open Web Text)

Max Activating Token Index: 345

Text #1

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4962. Min Act: -0.1700

Data Index: 2091077 (Open Web Text)

Max Activating Token Index: 444

Text #2

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4099. Min Act: -0.1700

Data Index: 2594215 (Open Web Text)

Max Activating Token Index: 555

Text #3

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4268. Min Act: -0.1700

Data Index: 2827170 (Open Web Text)

Max Activating Token Index: 526

Text #4

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3961. Min Act: -0.1700

Data Index: 419486 (Open Web Text)

Max Activating Token Index: 235

Text #5

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4137. Min Act: -0.1700

Data Index: 3440634 (Open Web Text)

Max Activating Token Index: 109

Text #6

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4060. Min Act: -0.1700

Data Index: 8457591 (Open Web Text)

Max Activating Token Index: 37

Text #7

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4280. Min Act: -0.1700

Data Index: 7493054 (Open Web Text)

Max Activating Token Index: 38

Text #8

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4148. Min Act: -0.1700

Data Index: 7176914 (Open Web Text)

Max Activating Token Index: 317

Text #9

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.4147. Min Act: -0.1700

Data Index: 1892384 (Open Web Text)

Max Activating Token Index: 435

Text #10

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3783. Min Act: -0.1700

Data Index: 3828245 (Open Web Text)

Max Activating Token Index: 11

Text #11

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3568. Min Act: -0.1700

Data Index: 5776623 (Open Web Text)

Max Activating Token Index: 27

Text #12

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3752. Min Act: -0.1700

Data Index: 5942912 (Open Web Text)

Max Activating Token Index: 483

Text #13

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3765. Min Act: -0.1700

Data Index: 3676223 (Open Web Text)

Max Activating Token Index: 55

Text #14

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3543. Min Act: -0.1700

Data Index: 5606541 (Open Web Text)

Max Activating Token Index: 50

Text #15

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3337. Min Act: -0.1700

Data Index: 936483 (Open Web Text)

Max Activating Token Index: 915

Text #16

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3613. Min Act: -0.1700

Data Index: 2290278 (Open Web Text)

Max Activating Token Index: 40

Text #17

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3478. Min Act: -0.1700

Data Index: 2565928 (Open Web Text)

Max Activating Token Index: 27

Text #18

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3610. Min Act: -0.1700

Data Index: 277245 (Open Web Text)

Max Activating Token Index: 517

Text #19

Max Range: 2.5334. Min Range: -2.5334

Max Act: 2.3477. Min Act: -0.1700

Data Index: 4798491 (Open Web Text)

Max Activating Token Index: 453