Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 11 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.7712. Min Act: -0.1700

Data Index: 6972709 (Open Web Text)

Max Activating Token Index: 773

Text #1

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.7459. Min Act: -0.1700

Data Index: 5620246 (Open Web Text)

Max Activating Token Index: 118

Text #2

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.7159. Min Act: -0.1700

Data Index: 7271580 (Open Web Text)

Max Activating Token Index: 670

Text #3

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6994. Min Act: -0.1700

Data Index: 3361467 (Open Web Text)

Max Activating Token Index: 416

Text #4

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6952. Min Act: -0.1700

Data Index: 3066989 (Open Web Text)

Max Activating Token Index: 581

Text #5

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6665. Min Act: -0.1700

Data Index: 2346901 (Open Web Text)

Max Activating Token Index: 417

Text #6

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6758. Min Act: -0.1700

Data Index: 6512354 (Open Web Text)

Max Activating Token Index: 56

Text #7

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6725. Min Act: -0.1700

Data Index: 1018772 (Open Web Text)

Max Activating Token Index: 728

Text #8

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6659. Min Act: -0.1700

Data Index: 7210565 (Open Web Text)

Max Activating Token Index: 348

Text #9

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6521. Min Act: -0.1700

Data Index: 4708488 (Open Web Text)

Max Activating Token Index: 129

Text #10

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6529. Min Act: -0.1700

Data Index: 100810 (Open Web Text)

Max Activating Token Index: 210

Text #11

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6494. Min Act: -0.1700

Data Index: 6846934 (Open Web Text)

Max Activating Token Index: 809

Text #12

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6549. Min Act: -0.1700

Data Index: 6745932 (Open Web Text)

Max Activating Token Index: 593

Text #13

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6370. Min Act: -0.1700

Data Index: 1968349 (Open Web Text)

Max Activating Token Index: 281

Text #14

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6350. Min Act: -0.1700

Data Index: 7055661 (Open Web Text)

Max Activating Token Index: 332

Text #15

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6321. Min Act: -0.1700

Data Index: 3320069 (Open Web Text)

Max Activating Token Index: 553

Text #16

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6160. Min Act: -0.1700

Data Index: 6921435 (Open Web Text)

Max Activating Token Index: 328

Text #17

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6193. Min Act: -0.1700

Data Index: 2048386 (Open Web Text)

Max Activating Token Index: 35

Text #18

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6187. Min Act: -0.1700

Data Index: 1195948 (Open Web Text)

Max Activating Token Index: 112

Text #19

Max Range: 1.7712. Min Range: -1.7712

Max Act: 1.6188. Min Act: -0.1698

Data Index: 3551397 (Open Web Text)

Max Activating Token Index: 249