Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1039 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0974. Min Range: -5.0974

Max Act: 5.0974. Min Act: -0.1700

Data Index: 464852 (Open Web Text)

Max Activating Token Index: 28

Text #1

Max Range: 5.0974. Min Range: -5.0974

Max Act: 5.0378. Min Act: -0.1700

Data Index: 4213931 (Open Web Text)

Max Activating Token Index: 240

Text #2

Max Range: 5.0974. Min Range: -5.0974

Max Act: 5.0372. Min Act: -0.1700

Data Index: 4469498 (Open Web Text)

Max Activating Token Index: 757

Text #3

Max Range: 5.0974. Min Range: -5.0974

Max Act: 5.0280. Min Act: -0.1700

Data Index: 430542 (Open Web Text)

Max Activating Token Index: 802

Text #4

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9551. Min Act: -0.1700

Data Index: 2917629 (Open Web Text)

Max Activating Token Index: 138

Text #5

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9793. Min Act: -0.1700

Data Index: 4777682 (Open Web Text)

Max Activating Token Index: 356

Text #6

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9161. Min Act: -0.1700

Data Index: 587774 (Open Web Text)

Max Activating Token Index: 125

Text #7

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9240. Min Act: -0.1700

Data Index: 1477659 (Open Web Text)

Max Activating Token Index: 49

Text #8

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8961. Min Act: -0.1700

Data Index: 2359822 (Open Web Text)

Max Activating Token Index: 30

Text #9

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9323. Min Act: -0.1700

Data Index: 519192 (Open Web Text)

Max Activating Token Index: 433

Text #10

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9320. Min Act: -0.1700

Data Index: 2287045 (Open Web Text)

Max Activating Token Index: 221

Text #11

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.9017. Min Act: -0.1700

Data Index: 7825448 (Open Web Text)

Max Activating Token Index: 115

Text #12

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8933. Min Act: -0.1700

Data Index: 6301353 (Open Web Text)

Max Activating Token Index: 302

Text #13

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8903. Min Act: -0.1700

Data Index: 4213932 (Open Web Text)

Max Activating Token Index: 55

Text #14

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8768. Min Act: -0.1700

Data Index: 4623201 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8625. Min Act: -0.1700

Data Index: 4537114 (Open Web Text)

Max Activating Token Index: 1004

Text #16

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8356. Min Act: -0.1700

Data Index: 5431259 (Open Web Text)

Max Activating Token Index: 602

Text #17

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8776. Min Act: -0.1700

Data Index: 6056401 (Open Web Text)

Max Activating Token Index: 17

Text #18

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8552. Min Act: -0.1700

Data Index: 6309567 (Open Web Text)

Max Activating Token Index: 231

Text #19

Max Range: 5.0974. Min Range: -5.0974

Max Act: 4.8823. Min Act: -0.1700

Data Index: 6608153 (Open Web Text)

Max Activating Token Index: 273