Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2410 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.8265. Min Act: -0.1697

Data Index: 798682 (Open Web Text)

Max Activating Token Index: 156

Text #1

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7740. Min Act: -0.1697

Data Index: 1673162 (Open Web Text)

Max Activating Token Index: 395

Text #2

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7600. Min Act: -0.1678

Data Index: 4687867 (Open Web Text)

Max Activating Token Index: 191

Text #3

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7521. Min Act: -0.1690

Data Index: 5968970 (Open Web Text)

Max Activating Token Index: 965

Text #4

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7447. Min Act: -0.1700

Data Index: 2223320 (Open Web Text)

Max Activating Token Index: 816

Text #5

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7452. Min Act: -0.1698

Data Index: 8059502 (Open Web Text)

Max Activating Token Index: 720

Text #6

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7301. Min Act: -0.1675

Data Index: 6421822 (Open Web Text)

Max Activating Token Index: 383

Text #7

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7181. Min Act: -0.1687

Data Index: 1604209 (Open Web Text)

Max Activating Token Index: 954

Text #8

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7198. Min Act: -0.1694

Data Index: 628577 (Open Web Text)

Max Activating Token Index: 736

Text #9

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7178. Min Act: -0.1700

Data Index: 7107269 (Open Web Text)

Max Activating Token Index: 372

Text #10

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7208. Min Act: -0.1692

Data Index: 4906348 (Open Web Text)

Max Activating Token Index: 378

Text #11

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7045. Min Act: -0.1675

Data Index: 7756471 (Open Web Text)

Max Activating Token Index: 479

Text #12

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7139. Min Act: -0.1700

Data Index: 5788267 (Open Web Text)

Max Activating Token Index: 572

Text #13

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7136. Min Act: -0.1700

Data Index: 1329873 (Open Web Text)

Max Activating Token Index: 355

Text #14

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7067. Min Act: -0.1700

Data Index: 4829853 (Open Web Text)

Max Activating Token Index: 547

Text #15

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7000. Min Act: -0.1700

Data Index: 1688720 (Open Web Text)

Max Activating Token Index: 886

Text #16

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7051. Min Act: -0.1682

Data Index: 1021086 (Open Web Text)

Max Activating Token Index: 381

Text #17

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.7021. Min Act: -0.1696

Data Index: 3552467 (Open Web Text)

Max Activating Token Index: 384

Text #18

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.6974. Min Act: -0.1700

Data Index: 1978683 (Open Web Text)

Max Activating Token Index: 402

Text #19

Max Range: 1.8265. Min Range: -1.8265

Max Act: 1.6878. Min Act: -0.1700

Data Index: 6781281 (Open Web Text)

Max Activating Token Index: 365