Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1838 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7900. Min Act: -0.1700

Data Index: 3007760 (Open Web Text)

Max Activating Token Index: 180

Text #1

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7858. Min Act: -0.1699

Data Index: 6471991 (Open Web Text)

Max Activating Token Index: 678

Text #2

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7717. Min Act: -0.1688

Data Index: 2935820 (Open Web Text)

Max Activating Token Index: 1004

Text #3

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7725. Min Act: -0.1700

Data Index: 2569328 (Open Web Text)

Max Activating Token Index: 857

Text #4

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7663. Min Act: -0.1695

Data Index: 7293270 (Open Web Text)

Max Activating Token Index: 808

Text #5

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7522. Min Act: -0.1699

Data Index: 3634675 (Open Web Text)

Max Activating Token Index: 752

Text #6

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7506. Min Act: -0.1700

Data Index: 1048113 (Open Web Text)

Max Activating Token Index: 526

Text #7

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7528. Min Act: -0.1692

Data Index: 6142488 (Open Web Text)

Max Activating Token Index: 206

Text #8

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7580. Min Act: -0.1700

Data Index: 2813216 (Open Web Text)

Max Activating Token Index: 216

Text #9

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7472. Min Act: -0.1641

Data Index: 3213143 (Open Web Text)

Max Activating Token Index: 150

Text #10

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7491. Min Act: -0.1700

Data Index: 5268136 (Open Web Text)

Max Activating Token Index: 636

Text #11

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7402. Min Act: -0.1678

Data Index: 6756824 (Open Web Text)

Max Activating Token Index: 508

Text #12

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7474. Min Act: -0.1698

Data Index: 4383145 (Open Web Text)

Max Activating Token Index: 855

Text #13

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7365. Min Act: -0.1700

Data Index: 5653019 (Open Web Text)

Max Activating Token Index: 527

Text #14

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7404. Min Act: -0.1700

Data Index: 2267329 (Open Web Text)

Max Activating Token Index: 463

Text #15

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7435. Min Act: -0.1698

Data Index: 6064075 (Open Web Text)

Max Activating Token Index: 259

Text #16

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7354. Min Act: -0.1700

Data Index: 6011235 (Open Web Text)

Max Activating Token Index: 628

Text #17

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7352. Min Act: -0.1700

Data Index: 7445473 (Open Web Text)

Max Activating Token Index: 688

Text #18

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7407. Min Act: -0.1688

Data Index: 3953030 (Open Web Text)

Max Activating Token Index: 281

Text #19

Max Range: 1.7900. Min Range: -1.7900

Max Act: 1.7386. Min Act: -0.1686

Data Index: 8265056 (Open Web Text)

Max Activating Token Index: 78