Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1351 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2494. Min Range: -4.2494

Max Act: 4.2494. Min Act: -0.1700

Data Index: 7932127 (Open Web Text)

Max Activating Token Index: 966

Text #1

Max Range: 4.2494. Min Range: -4.2494

Max Act: 4.2258. Min Act: -0.1700

Data Index: 4355869 (Open Web Text)

Max Activating Token Index: 647

Text #2

Max Range: 4.2494. Min Range: -4.2494

Max Act: 4.0184. Min Act: -0.1700

Data Index: 256726 (Open Web Text)

Max Activating Token Index: 84

Text #3

Max Range: 4.2494. Min Range: -4.2494

Max Act: 4.0263. Min Act: -0.1700

Data Index: 8142258 (Open Web Text)

Max Activating Token Index: 263

Text #4

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9645. Min Act: -0.1700

Data Index: 3812109 (Open Web Text)

Max Activating Token Index: 622

Text #5

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9874. Min Act: -0.1700

Data Index: 5019487 (Open Web Text)

Max Activating Token Index: 55

Text #6

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9214. Min Act: -0.1700

Data Index: 164667 (Open Web Text)

Max Activating Token Index: 993

Text #7

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9483. Min Act: -0.1700

Data Index: 3918655 (Open Web Text)

Max Activating Token Index: 816

Text #8

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9449. Min Act: -0.1700

Data Index: 4597148 (Open Web Text)

Max Activating Token Index: 216

Text #9

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9111. Min Act: -0.1700

Data Index: 4511887 (Open Web Text)

Max Activating Token Index: 466

Text #10

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9321. Min Act: -0.1700

Data Index: 7966672 (Open Web Text)

Max Activating Token Index: 906

Text #11

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9245. Min Act: -0.1700

Data Index: 3175632 (Open Web Text)

Max Activating Token Index: 478

Text #12

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9133. Min Act: -0.1700

Data Index: 3684656 (Open Web Text)

Max Activating Token Index: 829

Text #13

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9107. Min Act: -0.1700

Data Index: 2734092 (Open Web Text)

Max Activating Token Index: 376

Text #14

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9024. Min Act: -0.1700

Data Index: 7166493 (Open Web Text)

Max Activating Token Index: 603

Text #15

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9071. Min Act: -0.1700

Data Index: 4043170 (Open Web Text)

Max Activating Token Index: 844

Text #16

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9436. Min Act: -0.1700

Data Index: 4168774 (Open Web Text)

Max Activating Token Index: 934

Text #17

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.9038. Min Act: -0.1700

Data Index: 298446 (Open Web Text)

Max Activating Token Index: 214

Text #18

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.8781. Min Act: -0.1700

Data Index: 3968369 (Open Web Text)

Max Activating Token Index: 286

Text #19

Max Range: 4.2494. Min Range: -4.2494

Max Act: 3.8902. Min Act: -0.1700

Data Index: 4022431 (Open Web Text)

Max Activating Token Index: 422