Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2209 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.5649. Min Act: -0.1700

Data Index: 5237356 (Open Web Text)

Max Activating Token Index: 372

Text #1

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3810. Min Act: -0.1700

Data Index: 8269209 (Open Web Text)

Max Activating Token Index: 676

Text #2

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3358. Min Act: -0.1700

Data Index: 3560879 (Open Web Text)

Max Activating Token Index: 697

Text #3

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3100. Min Act: -0.1700

Data Index: 1598489 (Open Web Text)

Max Activating Token Index: 229

Text #4

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2939. Min Act: -0.1700

Data Index: 8352334 (Open Web Text)

Max Activating Token Index: 366

Text #5

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3184. Min Act: -0.1700

Data Index: 3856274 (Open Web Text)

Max Activating Token Index: 393

Text #6

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3078. Min Act: -0.1700

Data Index: 1128393 (Open Web Text)

Max Activating Token Index: 681

Text #7

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.3041. Min Act: -0.1700

Data Index: 844545 (Open Web Text)

Max Activating Token Index: 932

Text #8

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2749. Min Act: -0.1700

Data Index: 2947728 (Open Web Text)

Max Activating Token Index: 744

Text #9

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2581. Min Act: -0.1700

Data Index: 3368847 (Open Web Text)

Max Activating Token Index: 473

Text #10

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2567. Min Act: -0.1700

Data Index: 2185625 (Open Web Text)

Max Activating Token Index: 892

Text #11

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2538. Min Act: -0.1700

Data Index: 3588404 (Open Web Text)

Max Activating Token Index: 143

Text #12

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2524. Min Act: -0.1700

Data Index: 7211295 (Open Web Text)

Max Activating Token Index: 169

Text #13

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2410. Min Act: -0.1700

Data Index: 6366362 (Open Web Text)

Max Activating Token Index: 808

Text #14

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2249. Min Act: -0.1700

Data Index: 1620809 (Open Web Text)

Max Activating Token Index: 173

Text #15

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2370. Min Act: -0.1700

Data Index: 8283197 (Open Web Text)

Max Activating Token Index: 894

Text #16

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2362. Min Act: -0.1700

Data Index: 8190257 (Open Web Text)

Max Activating Token Index: 430

Text #17

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.2253. Min Act: -0.1700

Data Index: 6035240 (Open Web Text)

Max Activating Token Index: 873

Text #18

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.1921. Min Act: -0.1700

Data Index: 7008643 (Open Web Text)

Max Activating Token Index: 312

Text #19

Max Range: 3.5649. Min Range: -3.5649

Max Act: 3.1964. Min Act: -0.1700

Data Index: 2766591 (Open Web Text)

Max Activating Token Index: 866