Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2819 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0315. Min Range: -3.0315

Max Act: 3.0315. Min Act: -0.1700

Data Index: 5549001 (Open Web Text)

Max Activating Token Index: 443

Text #1

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.9722. Min Act: -0.1700

Data Index: 4711595 (Open Web Text)

Max Activating Token Index: 80

Text #2

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.9289. Min Act: -0.1700

Data Index: 4758334 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.9008. Min Act: -0.1700

Data Index: 8621430 (Open Web Text)

Max Activating Token Index: 292

Text #4

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8864. Min Act: -0.1700

Data Index: 6187155 (Open Web Text)

Max Activating Token Index: 241

Text #5

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8756. Min Act: -0.1700

Data Index: 7800903 (Open Web Text)

Max Activating Token Index: 931

Text #6

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8441. Min Act: -0.1700

Data Index: 8644624 (Open Web Text)

Max Activating Token Index: 628

Text #7

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8416. Min Act: -0.1700

Data Index: 5407832 (Open Web Text)

Max Activating Token Index: 844

Text #8

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8332. Min Act: -0.1700

Data Index: 8347925 (Open Web Text)

Max Activating Token Index: 150

Text #9

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8286. Min Act: -0.1700

Data Index: 3431289 (Open Web Text)

Max Activating Token Index: 21

Text #10

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8032. Min Act: -0.1700

Data Index: 1288762 (Open Web Text)

Max Activating Token Index: 285

Text #11

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8020. Min Act: -0.1700

Data Index: 5805640 (Open Web Text)

Max Activating Token Index: 1005

Text #12

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8110. Min Act: -0.1700

Data Index: 4876107 (Open Web Text)

Max Activating Token Index: 1013

Text #13

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8155. Min Act: -0.1700

Data Index: 350564 (Open Web Text)

Max Activating Token Index: 412

Text #14

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8181. Min Act: -0.1700

Data Index: 2620522 (Open Web Text)

Max Activating Token Index: 464

Text #15

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.7909. Min Act: -0.1700

Data Index: 8129119 (Open Web Text)

Max Activating Token Index: 70

Text #16

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8021. Min Act: -0.1700

Data Index: 6192399 (Open Web Text)

Max Activating Token Index: 593

Text #17

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8080. Min Act: -0.1700

Data Index: 4580366 (Open Web Text)

Max Activating Token Index: 975

Text #18

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8031. Min Act: -0.1700

Data Index: 2668671 (Open Web Text)

Max Activating Token Index: 28

Text #19

Max Range: 3.0315. Min Range: -3.0315

Max Act: 2.8107. Min Act: -0.1700

Data Index: 8618738 (Open Web Text)

Max Activating Token Index: 793