Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 170 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9917. Min Act: -0.1700

Data Index: 357878 (Open Web Text)

Max Activating Token Index: 417

Text #1

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9672. Min Act: -0.1700

Data Index: 6397262 (Open Web Text)

Max Activating Token Index: 417

Text #2

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9514. Min Act: -0.1700

Data Index: 1383617 (Open Web Text)

Max Activating Token Index: 212

Text #3

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9270. Min Act: -0.1700

Data Index: 1147659 (Open Web Text)

Max Activating Token Index: 781

Text #4

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9199. Min Act: -0.1700

Data Index: 1047846 (Open Web Text)

Max Activating Token Index: 510

Text #5

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9144. Min Act: -0.1700

Data Index: 3192864 (Open Web Text)

Max Activating Token Index: 939

Text #6

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8883. Min Act: -0.1700

Data Index: 1056465 (Open Web Text)

Max Activating Token Index: 972

Text #7

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8964. Min Act: -0.1700

Data Index: 511264 (Open Web Text)

Max Activating Token Index: 941

Text #8

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.9020. Min Act: -0.1700

Data Index: 3240104 (Open Web Text)

Max Activating Token Index: 196

Text #9

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8939. Min Act: -0.1700

Data Index: 2864273 (Open Web Text)

Max Activating Token Index: 618

Text #10

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8795. Min Act: -0.1700

Data Index: 1383616 (Open Web Text)

Max Activating Token Index: 377

Text #11

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8694. Min Act: -0.1700

Data Index: 3437294 (Open Web Text)

Max Activating Token Index: 416

Text #12

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8623. Min Act: -0.1700

Data Index: 6774893 (Open Web Text)

Max Activating Token Index: 81

Text #13

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8472. Min Act: -0.1700

Data Index: 8553288 (Open Web Text)

Max Activating Token Index: 72

Text #14

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8580. Min Act: -0.1700

Data Index: 1890615 (Open Web Text)

Max Activating Token Index: 421

Text #15

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8587. Min Act: -0.1700

Data Index: 1115564 (Open Web Text)

Max Activating Token Index: 410

Text #16

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8500. Min Act: -0.1700

Data Index: 6036530 (Open Web Text)

Max Activating Token Index: 178

Text #17

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8785. Min Act: -0.1700

Data Index: 340533 (Open Web Text)

Max Activating Token Index: 457

Text #18

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8569. Min Act: -0.1700

Data Index: 610802 (Open Web Text)

Max Activating Token Index: 104

Text #19

Max Range: 3.9917. Min Range: -3.9917

Max Act: 3.8426. Min Act: -0.1700

Data Index: 4834222 (Open Web Text)

Max Activating Token Index: 203