Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2076 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.7770. Min Act: -0.1700

Data Index: 909634 (Open Web Text)

Max Activating Token Index: 854

Text #1

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.7395. Min Act: -0.1700

Data Index: 462899 (Open Web Text)

Max Activating Token Index: 417

Text #2

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.6291. Min Act: -0.1700

Data Index: 4783223 (Open Web Text)

Max Activating Token Index: 927

Text #3

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5881. Min Act: -0.1700

Data Index: 7575868 (Open Web Text)

Max Activating Token Index: 370

Text #4

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5768. Min Act: -0.1700

Data Index: 3206147 (Open Web Text)

Max Activating Token Index: 101

Text #5

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5508. Min Act: -0.1700

Data Index: 4341510 (Open Web Text)

Max Activating Token Index: 60

Text #6

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5632. Min Act: -0.1700

Data Index: 4875441 (Open Web Text)

Max Activating Token Index: 296

Text #7

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5386. Min Act: -0.1700

Data Index: 4609244 (Open Web Text)

Max Activating Token Index: 754

Text #8

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5318. Min Act: -0.1700

Data Index: 5782976 (Open Web Text)

Max Activating Token Index: 524

Text #9

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5252. Min Act: -0.1700

Data Index: 7239502 (Open Web Text)

Max Activating Token Index: 702

Text #10

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5347. Min Act: -0.1700

Data Index: 3138273 (Open Web Text)

Max Activating Token Index: 99

Text #11

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5371. Min Act: -0.1700

Data Index: 1519196 (Open Web Text)

Max Activating Token Index: 54

Text #12

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5350. Min Act: -0.1700

Data Index: 800154 (Open Web Text)

Max Activating Token Index: 344

Text #13

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5124. Min Act: -0.1700

Data Index: 6162504 (Open Web Text)

Max Activating Token Index: 807

Text #14

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5206. Min Act: -0.1700

Data Index: 2291424 (Open Web Text)

Max Activating Token Index: 166

Text #15

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.4898. Min Act: -0.1700

Data Index: 2229902 (Open Web Text)

Max Activating Token Index: 529

Text #16

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5017. Min Act: -0.1700

Data Index: 4535061 (Open Web Text)

Max Activating Token Index: 10

Text #17

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.4938. Min Act: -0.1700

Data Index: 262228 (Open Web Text)

Max Activating Token Index: 945

Text #18

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.5038. Min Act: -0.1700

Data Index: 8267278 (Open Web Text)

Max Activating Token Index: 63

Text #19

Max Range: 3.7770. Min Range: -3.7770

Max Act: 3.4627. Min Act: -0.1700

Data Index: 7548943 (Open Web Text)

Max Activating Token Index: 405