Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 184 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.4277. Min Act: -0.1700

Data Index: 5082376 (Open Web Text)

Max Activating Token Index: 848

Text #1

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3579. Min Act: -0.1700

Data Index: 3313721 (Open Web Text)

Max Activating Token Index: 554

Text #2

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3472. Min Act: -0.1700

Data Index: 8375247 (Open Web Text)

Max Activating Token Index: 402

Text #3

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3149. Min Act: -0.1700

Data Index: 8779519 (Open Web Text)

Max Activating Token Index: 814

Text #4

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3099. Min Act: -0.1700

Data Index: 784106 (Open Web Text)

Max Activating Token Index: 652

Text #5

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3177. Min Act: -0.1700

Data Index: 739871 (Open Web Text)

Max Activating Token Index: 906

Text #6

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3047. Min Act: -0.1700

Data Index: 3960816 (Open Web Text)

Max Activating Token Index: 790

Text #7

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.3124. Min Act: -0.1700

Data Index: 4090726 (Open Web Text)

Max Activating Token Index: 528

Text #8

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2906. Min Act: -0.1700

Data Index: 8470178 (Open Web Text)

Max Activating Token Index: 53

Text #9

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2878. Min Act: -0.1700

Data Index: 5314944 (Open Web Text)

Max Activating Token Index: 125

Text #10

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2840. Min Act: -0.1700

Data Index: 3982598 (Open Web Text)

Max Activating Token Index: 1007

Text #11

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2599. Min Act: -0.1700

Data Index: 651033 (Open Web Text)

Max Activating Token Index: 384

Text #12

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2719. Min Act: -0.1700

Data Index: 2350840 (Open Web Text)

Max Activating Token Index: 1018

Text #13

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2765. Min Act: -0.1700

Data Index: 8121623 (Open Web Text)

Max Activating Token Index: 965

Text #14

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2573. Min Act: -0.1700

Data Index: 1434623 (Open Web Text)

Max Activating Token Index: 710

Text #15

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2629. Min Act: -0.1700

Data Index: 7353670 (Open Web Text)

Max Activating Token Index: 1005

Text #16

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2487. Min Act: -0.1700

Data Index: 7246004 (Open Web Text)

Max Activating Token Index: 424

Text #17

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2539. Min Act: -0.1700

Data Index: 1713531 (Open Web Text)

Max Activating Token Index: 650

Text #18

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2436. Min Act: -0.1700

Data Index: 6840321 (Open Web Text)

Max Activating Token Index: 1012

Text #19

Max Range: 3.4277. Min Range: -3.4277

Max Act: 3.2497. Min Act: -0.1700

Data Index: 6134368 (Open Web Text)

Max Activating Token Index: 865