Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1075 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.8152. Min Act: -0.1700

Data Index: 4564572 (Open Web Text)

Max Activating Token Index: 206

Text #1

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.9493. Min Act: -0.1700

Data Index: 2129334 (Open Web Text)

Max Activating Token Index: 390

Text #2

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.8156. Min Act: -0.1694

Data Index: 7969307 (Open Web Text)

Max Activating Token Index: 223

Text #3

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7594. Min Act: -0.1699

Data Index: 64640 (Open Web Text)

Max Activating Token Index: 976

Text #4

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.8151. Min Act: -0.1700

Data Index: 7684814 (Open Web Text)

Max Activating Token Index: 452

Text #5

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.8025. Min Act: -0.1700

Data Index: 1301701 (Open Web Text)

Max Activating Token Index: 900

Text #6

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7656. Min Act: -0.1698

Data Index: 2143867 (Open Web Text)

Max Activating Token Index: 784

Text #7

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7265. Min Act: -0.1700

Data Index: 4255806 (Open Web Text)

Max Activating Token Index: 326

Text #8

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7956. Min Act: -0.1700

Data Index: 92732 (Open Web Text)

Max Activating Token Index: 646

Text #9

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7722. Min Act: -0.1693

Data Index: 7979737 (Open Web Text)

Max Activating Token Index: 265

Text #10

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7154. Min Act: -0.1700

Data Index: 1496750 (Open Web Text)

Max Activating Token Index: 278

Text #11

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.6943. Min Act: -0.1696

Data Index: 7694326 (Open Web Text)

Max Activating Token Index: 875

Text #12

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7178. Min Act: -0.1700

Data Index: 5545583 (Open Web Text)

Max Activating Token Index: 339

Text #13

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7478. Min Act: -0.1698

Data Index: 5888100 (Open Web Text)

Max Activating Token Index: 466

Text #14

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7380. Min Act: -0.1700

Data Index: 5231295 (Open Web Text)

Max Activating Token Index: 322

Text #15

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.7198. Min Act: -0.1700

Data Index: 6362205 (Open Web Text)

Max Activating Token Index: 783

Text #16

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.6872. Min Act: -0.1691

Data Index: 2340440 (Open Web Text)

Max Activating Token Index: 814

Text #17

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.6925. Min Act: -0.1700

Data Index: 3974961 (Open Web Text)

Max Activating Token Index: 992

Text #18

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.6603. Min Act: -0.1700

Data Index: 782104 (Open Web Text)

Max Activating Token Index: 696

Text #19

Max Range: 4.9493. Min Range: -4.9493

Max Act: 4.6583. Min Act: -0.1700

Data Index: 4214657 (Open Web Text)

Max Activating Token Index: 947