Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1403 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.8895. Min Act: -0.1700

Data Index: 36829 (Open Web Text)

Max Activating Token Index: 285

Text #1

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.8155. Min Act: -0.1700

Data Index: 1314475 (Open Web Text)

Max Activating Token Index: 441

Text #2

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7954. Min Act: -0.1700

Data Index: 4257426 (Open Web Text)

Max Activating Token Index: 620

Text #3

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7804. Min Act: -0.1700

Data Index: 7773475 (Open Web Text)

Max Activating Token Index: 577

Text #4

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7554. Min Act: -0.1700

Data Index: 2580359 (Open Web Text)

Max Activating Token Index: 28

Text #5

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7524. Min Act: -0.1700

Data Index: 919399 (Open Web Text)

Max Activating Token Index: 562

Text #6

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7618. Min Act: -0.1700

Data Index: 4183513 (Open Web Text)

Max Activating Token Index: 38

Text #7

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7229. Min Act: -0.1700

Data Index: 7176039 (Open Web Text)

Max Activating Token Index: 243

Text #8

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7397. Min Act: -0.1700

Data Index: 5251454 (Open Web Text)

Max Activating Token Index: 552

Text #9

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7224. Min Act: -0.1700

Data Index: 3860173 (Open Web Text)

Max Activating Token Index: 712

Text #10

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7047. Min Act: -0.1700

Data Index: 1787077 (Open Web Text)

Max Activating Token Index: 26

Text #11

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7357. Min Act: -0.1700

Data Index: 3388104 (Open Web Text)

Max Activating Token Index: 140

Text #12

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6746. Min Act: -0.1700

Data Index: 4147192 (Open Web Text)

Max Activating Token Index: 502

Text #13

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6822. Min Act: -0.1700

Data Index: 1372046 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6683. Min Act: -0.1700

Data Index: 1198623 (Open Web Text)

Max Activating Token Index: 462

Text #15

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6799. Min Act: -0.1700

Data Index: 4413174 (Open Web Text)

Max Activating Token Index: 906

Text #16

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6922. Min Act: -0.1700

Data Index: 4903513 (Open Web Text)

Max Activating Token Index: 301

Text #17

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.6923. Min Act: -0.1700

Data Index: 1696720 (Open Web Text)

Max Activating Token Index: 81

Text #18

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7073. Min Act: -0.1700

Data Index: 5077589 (Open Web Text)

Max Activating Token Index: 571

Text #19

Max Range: 4.8895. Min Range: -4.8895

Max Act: 4.7358. Min Act: -0.1700

Data Index: 5125661 (Open Web Text)

Max Activating Token Index: 895