Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3019 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.4384. Min Act: -0.1700

Data Index: 1843186 (Open Web Text)

Max Activating Token Index: 137

Text #1

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.4096. Min Act: -0.1700

Data Index: 7663463 (Open Web Text)

Max Activating Token Index: 835

Text #2

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3810. Min Act: -0.1700

Data Index: 6329990 (Open Web Text)

Max Activating Token Index: 665

Text #3

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3495. Min Act: -0.1700

Data Index: 1482287 (Open Web Text)

Max Activating Token Index: 211

Text #4

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3605. Min Act: -0.1700

Data Index: 8740809 (Open Web Text)

Max Activating Token Index: 972

Text #5

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3139. Min Act: -0.1700

Data Index: 2260458 (Open Web Text)

Max Activating Token Index: 520

Text #6

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2953. Min Act: -0.1700

Data Index: 2350872 (Open Web Text)

Max Activating Token Index: 141

Text #7

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3031. Min Act: -0.1700

Data Index: 8177840 (Open Web Text)

Max Activating Token Index: 380

Text #8

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.3128. Min Act: -0.1700

Data Index: 4590094 (Open Web Text)

Max Activating Token Index: 341

Text #9

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2587. Min Act: -0.1700

Data Index: 6574285 (Open Web Text)

Max Activating Token Index: 582

Text #10

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2730. Min Act: -0.1700

Data Index: 220216 (Open Web Text)

Max Activating Token Index: 15

Text #11

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2372. Min Act: -0.1700

Data Index: 8701466 (Open Web Text)

Max Activating Token Index: 364

Text #12

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2440. Min Act: -0.1700

Data Index: 2341862 (Open Web Text)

Max Activating Token Index: 282

Text #13

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2038. Min Act: -0.1700

Data Index: 7993167 (Open Web Text)

Max Activating Token Index: 817

Text #14

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2726. Min Act: -0.1700

Data Index: 4394430 (Open Web Text)

Max Activating Token Index: 878

Text #15

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2257. Min Act: -0.1700

Data Index: 442422 (Open Web Text)

Max Activating Token Index: 673

Text #16

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.1906. Min Act: -0.1700

Data Index: 7055360 (Open Web Text)

Max Activating Token Index: 544

Text #17

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.2043. Min Act: -0.1700

Data Index: 642431 (Open Web Text)

Max Activating Token Index: 893

Text #18

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.1911. Min Act: -0.1700

Data Index: 2751373 (Open Web Text)

Max Activating Token Index: 140

Text #19

Max Range: 4.4384. Min Range: -4.4384

Max Act: 4.1676. Min Act: -0.1700

Data Index: 3019822 (Open Web Text)

Max Activating Token Index: 742