Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1749 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.1270. Min Act: -0.1700

Data Index: 4304988 (Open Web Text)

Max Activating Token Index: 628

Text #1

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.1137. Min Act: -0.1700

Data Index: 466165 (Open Web Text)

Max Activating Token Index: 462

Text #2

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0875. Min Act: -0.1700

Data Index: 6592852 (Open Web Text)

Max Activating Token Index: 813

Text #3

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0918. Min Act: -0.1700

Data Index: 1654462 (Open Web Text)

Max Activating Token Index: 634

Text #4

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0975. Min Act: -0.1700

Data Index: 238239 (Open Web Text)

Max Activating Token Index: 841

Text #5

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0733. Min Act: -0.1700

Data Index: 4870025 (Open Web Text)

Max Activating Token Index: 644

Text #6

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0545. Min Act: -0.1700

Data Index: 7038574 (Open Web Text)

Max Activating Token Index: 102

Text #7

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0395. Min Act: -0.1700

Data Index: 6805429 (Open Web Text)

Max Activating Token Index: 901

Text #8

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0455. Min Act: -0.1700

Data Index: 3256626 (Open Web Text)

Max Activating Token Index: 801

Text #9

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0397. Min Act: -0.1700

Data Index: 7054794 (Open Web Text)

Max Activating Token Index: 194

Text #10

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0409. Min Act: -0.1700

Data Index: 3896288 (Open Web Text)

Max Activating Token Index: 524

Text #11

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0381. Min Act: -0.1700

Data Index: 8211807 (Open Web Text)

Max Activating Token Index: 483

Text #12

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0402. Min Act: -0.1700

Data Index: 5332161 (Open Web Text)

Max Activating Token Index: 678

Text #13

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0387. Min Act: -0.1700

Data Index: 4582013 (Open Web Text)

Max Activating Token Index: 619

Text #14

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0463. Min Act: -0.1700

Data Index: 1711394 (Open Web Text)

Max Activating Token Index: 844

Text #15

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0598. Min Act: -0.1700

Data Index: 514450 (Open Web Text)

Max Activating Token Index: 873

Text #16

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0325. Min Act: -0.1700

Data Index: 5057947 (Open Web Text)

Max Activating Token Index: 119

Text #17

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0334. Min Act: -0.1700

Data Index: 5581275 (Open Web Text)

Max Activating Token Index: 417

Text #18

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0479. Min Act: -0.1700

Data Index: 4021695 (Open Web Text)

Max Activating Token Index: 445

Text #19

Max Range: 2.1270. Min Range: -2.1270

Max Act: 2.0152. Min Act: -0.1700

Data Index: 1778016 (Open Web Text)

Max Activating Token Index: 568