Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 102 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.8490. Min Act: -0.1700

Data Index: 2121774 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.8324. Min Act: -0.1700

Data Index: 1571030 (Open Web Text)

Max Activating Token Index: 35

Text #2

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.8166. Min Act: -0.1700

Data Index: 2971328 (Open Web Text)

Max Activating Token Index: 510

Text #3

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.8107. Min Act: -0.1700

Data Index: 8002063 (Open Web Text)

Max Activating Token Index: 700

Text #4

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.8019. Min Act: -0.1700

Data Index: 7254295 (Open Web Text)

Max Activating Token Index: 433

Text #5

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7968. Min Act: -0.1700

Data Index: 8455639 (Open Web Text)

Max Activating Token Index: 860

Text #6

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7866. Min Act: -0.1700

Data Index: 3445709 (Open Web Text)

Max Activating Token Index: 60

Text #7

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7958. Min Act: -0.1700

Data Index: 8538136 (Open Web Text)

Max Activating Token Index: 237

Text #8

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7812. Min Act: -0.1699

Data Index: 183724 (Open Web Text)

Max Activating Token Index: 984

Text #9

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7915. Min Act: -0.1700

Data Index: 6209265 (Open Web Text)

Max Activating Token Index: 340

Text #10

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7981. Min Act: -0.1700

Data Index: 3099262 (Open Web Text)

Max Activating Token Index: 782

Text #11

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7793. Min Act: -0.1700

Data Index: 791338 (Open Web Text)

Max Activating Token Index: 40

Text #12

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7694. Min Act: -0.1700

Data Index: 6524856 (Open Web Text)

Max Activating Token Index: 252

Text #13

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7690. Min Act: -0.1700

Data Index: 3355109 (Open Web Text)

Max Activating Token Index: 104

Text #14

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7756. Min Act: -0.1700

Data Index: 1356812 (Open Web Text)

Max Activating Token Index: 507

Text #15

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7668. Min Act: -0.1700

Data Index: 1856202 (Open Web Text)

Max Activating Token Index: 630

Text #16

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7595. Min Act: -0.1700

Data Index: 8111406 (Open Web Text)

Max Activating Token Index: 562

Text #17

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7616. Min Act: -0.1700

Data Index: 4468517 (Open Web Text)

Max Activating Token Index: 562

Text #18

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7596. Min Act: -0.1700

Data Index: 7623682 (Open Web Text)

Max Activating Token Index: 873

Text #19

Max Range: 1.8490. Min Range: -1.8490

Max Act: 1.7688. Min Act: -0.1700

Data Index: 7303034 (Open Web Text)

Max Activating Token Index: 400