Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1033 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.4221. Min Act: -0.1700

Data Index: 1911225 (Open Web Text)

Max Activating Token Index: 525

Text #1

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3964. Min Act: -0.1700

Data Index: 4995240 (Open Web Text)

Max Activating Token Index: 616

Text #2

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3321. Min Act: -0.1700

Data Index: 1957610 (Open Web Text)

Max Activating Token Index: 377

Text #3

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3382. Min Act: -0.1700

Data Index: 890009 (Open Web Text)

Max Activating Token Index: 618

Text #4

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3406. Min Act: -0.1696

Data Index: 1538200 (Open Web Text)

Max Activating Token Index: 103

Text #5

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3337. Min Act: -0.1699

Data Index: 4995239 (Open Web Text)

Max Activating Token Index: 353

Text #6

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3296. Min Act: -0.1700

Data Index: 3001580 (Open Web Text)

Max Activating Token Index: 964

Text #7

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3340. Min Act: -0.1646

Data Index: 7182120 (Open Web Text)

Max Activating Token Index: 601

Text #8

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3300. Min Act: -0.1693

Data Index: 7182121 (Open Web Text)

Max Activating Token Index: 826

Text #9

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3221. Min Act: -0.1698

Data Index: 1980915 (Open Web Text)

Max Activating Token Index: 730

Text #10

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3004. Min Act: -0.1672

Data Index: 6085049 (Open Web Text)

Max Activating Token Index: 1018

Text #11

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.3118. Min Act: -0.1700

Data Index: 8265492 (Open Web Text)

Max Activating Token Index: 439

Text #12

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2966. Min Act: -0.1700

Data Index: 1068788 (Open Web Text)

Max Activating Token Index: 184

Text #13

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2957. Min Act: -0.1690

Data Index: 3191802 (Open Web Text)

Max Activating Token Index: 709

Text #14

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2822. Min Act: -0.1678

Data Index: 7915551 (Open Web Text)

Max Activating Token Index: 385

Text #15

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2964. Min Act: -0.1699

Data Index: 1978747 (Open Web Text)

Max Activating Token Index: 541

Text #16

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2972. Min Act: -0.1687

Data Index: 390044 (Open Web Text)

Max Activating Token Index: 443

Text #17

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2874. Min Act: -0.1700

Data Index: 3902851 (Open Web Text)

Max Activating Token Index: 550

Text #18

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2895. Min Act: -0.1700

Data Index: 4377694 (Open Web Text)

Max Activating Token Index: 384

Text #19

Max Range: 1.4221. Min Range: -1.4221

Max Act: 1.2780. Min Act: -0.1700

Data Index: 1221291 (Open Web Text)

Max Activating Token Index: 278