Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1682 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.6762. Min Act: -0.1700

Data Index: 7077771 (Open Web Text)

Max Activating Token Index: 138

Text #1

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.6859. Min Act: -0.1700

Data Index: 273169 (Open Web Text)

Max Activating Token Index: 58

Text #2

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.6087. Min Act: -0.1700

Data Index: 7120477 (Open Web Text)

Max Activating Token Index: 351

Text #3

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5970. Min Act: -0.1700

Data Index: 2550030 (Open Web Text)

Max Activating Token Index: 471

Text #4

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5856. Min Act: -0.1700

Data Index: 8197279 (Open Web Text)

Max Activating Token Index: 456

Text #5

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5638. Min Act: -0.1700

Data Index: 5168673 (Open Web Text)

Max Activating Token Index: 488

Text #6

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5731. Min Act: -0.1700

Data Index: 8012851 (Open Web Text)

Max Activating Token Index: 534

Text #7

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5472. Min Act: -0.1700

Data Index: 3752511 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5594. Min Act: -0.1700

Data Index: 6043415 (Open Web Text)

Max Activating Token Index: 186

Text #9

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5513. Min Act: -0.1700

Data Index: 4284409 (Open Web Text)

Max Activating Token Index: 68

Text #10

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5553. Min Act: -0.1700

Data Index: 6371314 (Open Web Text)

Max Activating Token Index: 146

Text #11

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5514. Min Act: -0.1700

Data Index: 8482402 (Open Web Text)

Max Activating Token Index: 180

Text #12

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5429. Min Act: -0.1700

Data Index: 7454143 (Open Web Text)

Max Activating Token Index: 410

Text #13

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5470. Min Act: -0.1700

Data Index: 7972 (Open Web Text)

Max Activating Token Index: 755

Text #14

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5331. Min Act: -0.1700

Data Index: 1915210 (Open Web Text)

Max Activating Token Index: 668

Text #15

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5082. Min Act: -0.1700

Data Index: 6711417 (Open Web Text)

Max Activating Token Index: 125

Text #16

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5165. Min Act: -0.1700

Data Index: 4303044 (Open Web Text)

Max Activating Token Index: 467

Text #17

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5241. Min Act: -0.1700

Data Index: 5268199 (Open Web Text)

Max Activating Token Index: 1008

Text #18

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.4899. Min Act: -0.1700

Data Index: 7364098 (Open Web Text)

Max Activating Token Index: 436

Text #19

Max Range: 3.6859. Min Range: -3.6859

Max Act: 3.5061. Min Act: -0.1700

Data Index: 3305584 (Open Web Text)

Max Activating Token Index: 631