Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1521 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.8613. Min Act: -0.1700

Data Index: 516401 (Open Web Text)

Max Activating Token Index: 872

Text #1

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.8171. Min Act: -0.1700

Data Index: 728086 (Open Web Text)

Max Activating Token Index: 140

Text #2

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.8026. Min Act: -0.1696

Data Index: 6972027 (Open Web Text)

Max Activating Token Index: 158

Text #3

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7869. Min Act: -0.1700

Data Index: 3055460 (Open Web Text)

Max Activating Token Index: 510

Text #4

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7870. Min Act: -0.1700

Data Index: 4679553 (Open Web Text)

Max Activating Token Index: 77

Text #5

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7750. Min Act: -0.1699

Data Index: 5195238 (Open Web Text)

Max Activating Token Index: 592

Text #6

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7728. Min Act: -0.1700

Data Index: 5066115 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7806. Min Act: -0.1700

Data Index: 4184934 (Open Web Text)

Max Activating Token Index: 525

Text #8

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7799. Min Act: -0.1700

Data Index: 3422762 (Open Web Text)

Max Activating Token Index: 755

Text #9

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7719. Min Act: -0.1700

Data Index: 5210395 (Open Web Text)

Max Activating Token Index: 1013

Text #10

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7549. Min Act: -0.1700

Data Index: 5873265 (Open Web Text)

Max Activating Token Index: 74

Text #11

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7481. Min Act: -0.1700

Data Index: 3360164 (Open Web Text)

Max Activating Token Index: 790

Text #12

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7655. Min Act: -0.1700

Data Index: 8831722 (Open Web Text)

Max Activating Token Index: 548

Text #13

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7408. Min Act: -0.1700

Data Index: 4165960 (Open Web Text)

Max Activating Token Index: 236

Text #14

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7524. Min Act: -0.1700

Data Index: 4916163 (Open Web Text)

Max Activating Token Index: 49

Text #15

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7551. Min Act: -0.1700

Data Index: 8697466 (Open Web Text)

Max Activating Token Index: 839

Text #16

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7429. Min Act: -0.1700

Data Index: 4129228 (Open Web Text)

Max Activating Token Index: 53

Text #17

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7474. Min Act: -0.1700

Data Index: 2701805 (Open Web Text)

Max Activating Token Index: 62

Text #18

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7565. Min Act: -0.1700

Data Index: 1221717 (Open Web Text)

Max Activating Token Index: 58

Text #19

Max Range: 1.8613. Min Range: -1.8613

Max Act: 1.7477. Min Act: -0.1700

Data Index: 4933811 (Open Web Text)

Max Activating Token Index: 59