Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1300 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.3099. Min Act: -0.1700

Data Index: 8390744 (Open Web Text)

Max Activating Token Index: 523

Text #1

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2497. Min Act: -0.1700

Data Index: 8241802 (Open Web Text)

Max Activating Token Index: 613

Text #2

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2309. Min Act: -0.1700

Data Index: 5191417 (Open Web Text)

Max Activating Token Index: 911

Text #3

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2595. Min Act: -0.1700

Data Index: 1682364 (Open Web Text)

Max Activating Token Index: 663

Text #4

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2108. Min Act: -0.1700

Data Index: 3857698 (Open Web Text)

Max Activating Token Index: 206

Text #5

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2011. Min Act: -0.1700

Data Index: 5011284 (Open Web Text)

Max Activating Token Index: 693

Text #6

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1823. Min Act: -0.1700

Data Index: 6049824 (Open Web Text)

Max Activating Token Index: 1012

Text #7

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1654. Min Act: -0.1700

Data Index: 3102582 (Open Web Text)

Max Activating Token Index: 152

Text #8

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.2000. Min Act: -0.1700

Data Index: 8630891 (Open Web Text)

Max Activating Token Index: 813

Text #9

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1566. Min Act: -0.1700

Data Index: 605637 (Open Web Text)

Max Activating Token Index: 860

Text #10

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1437. Min Act: -0.1700

Data Index: 3165263 (Open Web Text)

Max Activating Token Index: 498

Text #11

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1475. Min Act: -0.1700

Data Index: 4185952 (Open Web Text)

Max Activating Token Index: 695

Text #12

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1700. Min Act: -0.1700

Data Index: 3004513 (Open Web Text)

Max Activating Token Index: 435

Text #13

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1397. Min Act: -0.1700

Data Index: 8196815 (Open Web Text)

Max Activating Token Index: 308

Text #14

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1169. Min Act: -0.1700

Data Index: 150214 (Open Web Text)

Max Activating Token Index: 433

Text #15

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1402. Min Act: -0.1700

Data Index: 2203817 (Open Web Text)

Max Activating Token Index: 839

Text #16

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1263. Min Act: -0.1700

Data Index: 358729 (Open Web Text)

Max Activating Token Index: 342

Text #17

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.1127. Min Act: -0.1700

Data Index: 4030836 (Open Web Text)

Max Activating Token Index: 333

Text #18

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.0945. Min Act: -0.1700

Data Index: 3765158 (Open Web Text)

Max Activating Token Index: 757

Text #19

Max Range: 4.3099. Min Range: -4.3099

Max Act: 4.0927. Min Act: -0.1700

Data Index: 5275701 (Open Web Text)

Max Activating Token Index: 183