Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 764 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0280. Min Range: -3.0280

Max Act: 3.0280. Min Act: -0.1700

Data Index: 6200735 (Open Web Text)

Max Activating Token Index: 242

Text #1

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.9115. Min Act: -0.1700

Data Index: 277169 (Open Web Text)

Max Activating Token Index: 152

Text #2

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8450. Min Act: -0.1700

Data Index: 5133628 (Open Web Text)

Max Activating Token Index: 736

Text #3

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8876. Min Act: -0.1700

Data Index: 7015643 (Open Web Text)

Max Activating Token Index: 874

Text #4

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8449. Min Act: -0.1700

Data Index: 8344477 (Open Web Text)

Max Activating Token Index: 865

Text #5

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8346. Min Act: -0.1700

Data Index: 2319863 (Open Web Text)

Max Activating Token Index: 362

Text #6

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8303. Min Act: -0.1700

Data Index: 2329606 (Open Web Text)

Max Activating Token Index: 913

Text #7

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8074. Min Act: -0.1700

Data Index: 72807 (Open Web Text)

Max Activating Token Index: 736

Text #8

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8264. Min Act: -0.1700

Data Index: 5306222 (Open Web Text)

Max Activating Token Index: 820

Text #9

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7979. Min Act: -0.1700

Data Index: 4854415 (Open Web Text)

Max Activating Token Index: 402

Text #10

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.8053. Min Act: -0.1700

Data Index: 3885042 (Open Web Text)

Max Activating Token Index: 566

Text #11

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7926. Min Act: -0.1700

Data Index: 4583843 (Open Web Text)

Max Activating Token Index: 518

Text #12

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7734. Min Act: -0.1700

Data Index: 1945146 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7537. Min Act: -0.1700

Data Index: 2201778 (Open Web Text)

Max Activating Token Index: 433

Text #14

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7540. Min Act: -0.1700

Data Index: 7858169 (Open Web Text)

Max Activating Token Index: 904

Text #15

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7517. Min Act: -0.1700

Data Index: 1305191 (Open Web Text)

Max Activating Token Index: 559

Text #16

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7150. Min Act: -0.1700

Data Index: 419598 (Open Web Text)

Max Activating Token Index: 191

Text #17

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7376. Min Act: -0.1700

Data Index: 406175 (Open Web Text)

Max Activating Token Index: 75

Text #18

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7273. Min Act: -0.1700

Data Index: 3312182 (Open Web Text)

Max Activating Token Index: 524

Text #19

Max Range: 3.0280. Min Range: -3.0280

Max Act: 2.7245. Min Act: -0.1700

Data Index: 8658644 (Open Web Text)

Max Activating Token Index: 886