Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1247 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.7835. Min Act: -0.1700

Data Index: 5292180 (Open Web Text)

Max Activating Token Index: 672

Text #1

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.7765. Min Act: -0.1700

Data Index: 7105338 (Open Web Text)

Max Activating Token Index: 225

Text #2

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5528. Min Act: -0.1700

Data Index: 1928884 (Open Web Text)

Max Activating Token Index: 872

Text #3

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5741. Min Act: -0.1700

Data Index: 1608283 (Open Web Text)

Max Activating Token Index: 268

Text #4

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5777. Min Act: -0.1700

Data Index: 7004918 (Open Web Text)

Max Activating Token Index: 598

Text #5

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5847. Min Act: -0.1700

Data Index: 996464 (Open Web Text)

Max Activating Token Index: 323

Text #6

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5233. Min Act: -0.1700

Data Index: 6828613 (Open Web Text)

Max Activating Token Index: 125

Text #7

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5294. Min Act: -0.1700

Data Index: 7427060 (Open Web Text)

Max Activating Token Index: 800

Text #8

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5357. Min Act: -0.1700

Data Index: 1291938 (Open Web Text)

Max Activating Token Index: 838

Text #9

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4784. Min Act: -0.1700

Data Index: 4314247 (Open Web Text)

Max Activating Token Index: 332

Text #10

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4846. Min Act: -0.1700

Data Index: 8775118 (Open Web Text)

Max Activating Token Index: 543

Text #11

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4981. Min Act: -0.1700

Data Index: 258086 (Open Web Text)

Max Activating Token Index: 640

Text #12

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4846. Min Act: -0.1700

Data Index: 3795742 (Open Web Text)

Max Activating Token Index: 960

Text #13

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4659. Min Act: -0.1700

Data Index: 8783488 (Open Web Text)

Max Activating Token Index: 554

Text #14

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.5066. Min Act: -0.1700

Data Index: 489369 (Open Web Text)

Max Activating Token Index: 319

Text #15

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4821. Min Act: -0.1700

Data Index: 7825859 (Open Web Text)

Max Activating Token Index: 1014

Text #16

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4966. Min Act: -0.1700

Data Index: 4295136 (Open Web Text)

Max Activating Token Index: 707

Text #17

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4492. Min Act: -0.1700

Data Index: 1057290 (Open Web Text)

Max Activating Token Index: 980

Text #18

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4410. Min Act: -0.1700

Data Index: 1815664 (Open Web Text)

Max Activating Token Index: 52

Text #19

Max Range: 4.7835. Min Range: -4.7835

Max Act: 4.4537. Min Act: -0.1700

Data Index: 3960308 (Open Web Text)

Max Activating Token Index: 409