Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 434 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.1531. Min Act: -0.1700

Data Index: 8388422 (Open Web Text)

Max Activating Token Index: 81

Text #1

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0663. Min Act: -0.1700

Data Index: 939262 (Open Web Text)

Max Activating Token Index: 353

Text #2

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0894. Min Act: -0.1700

Data Index: 4267826 (Open Web Text)

Max Activating Token Index: 106

Text #3

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0624. Min Act: -0.1700

Data Index: 811744 (Open Web Text)

Max Activating Token Index: 218

Text #4

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0825. Min Act: -0.1700

Data Index: 7866329 (Open Web Text)

Max Activating Token Index: 446

Text #5

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0656. Min Act: -0.1700

Data Index: 385729 (Open Web Text)

Max Activating Token Index: 29

Text #6

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0583. Min Act: -0.1700

Data Index: 1848396 (Open Web Text)

Max Activating Token Index: 29

Text #7

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0473. Min Act: -0.1700

Data Index: 4806804 (Open Web Text)

Max Activating Token Index: 136

Text #8

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0166. Min Act: -0.1700

Data Index: 1548698 (Open Web Text)

Max Activating Token Index: 94

Text #9

Max Range: 5.1531. Min Range: -5.1531

Max Act: 5.0032. Min Act: -0.1700

Data Index: 8618859 (Open Web Text)

Max Activating Token Index: 48

Text #10

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9740. Min Act: -0.1700

Data Index: 5626644 (Open Web Text)

Max Activating Token Index: 940

Text #11

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9967. Min Act: -0.1700

Data Index: 901400 (Open Web Text)

Max Activating Token Index: 428

Text #12

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9706. Min Act: -0.1700

Data Index: 8755207 (Open Web Text)

Max Activating Token Index: 83

Text #13

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9560. Min Act: -0.1700

Data Index: 557418 (Open Web Text)

Max Activating Token Index: 89

Text #14

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9602. Min Act: -0.1700

Data Index: 5212957 (Open Web Text)

Max Activating Token Index: 142

Text #15

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9988. Min Act: -0.1700

Data Index: 519314 (Open Web Text)

Max Activating Token Index: 95

Text #16

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9714. Min Act: -0.1700

Data Index: 3276632 (Open Web Text)

Max Activating Token Index: 468

Text #17

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9782. Min Act: -0.1700

Data Index: 8079446 (Open Web Text)

Max Activating Token Index: 130

Text #18

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9924. Min Act: -0.1700

Data Index: 845100 (Open Web Text)

Max Activating Token Index: 461

Text #19

Max Range: 5.1531. Min Range: -5.1531

Max Act: 4.9592. Min Act: -0.1700

Data Index: 6131575 (Open Web Text)

Max Activating Token Index: 57