Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1687 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.8549. Min Act: -0.1700

Data Index: 2546461 (Open Web Text)

Max Activating Token Index: 253

Text #1

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.8373. Min Act: -0.1700

Data Index: 1042238 (Open Web Text)

Max Activating Token Index: 676

Text #2

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7587. Min Act: -0.1700

Data Index: 7310489 (Open Web Text)

Max Activating Token Index: 884

Text #3

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7625. Min Act: -0.1700

Data Index: 6995029 (Open Web Text)

Max Activating Token Index: 17

Text #4

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7655. Min Act: -0.1700

Data Index: 6585590 (Open Web Text)

Max Activating Token Index: 172

Text #5

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7693. Min Act: -0.1700

Data Index: 7765973 (Open Web Text)

Max Activating Token Index: 570

Text #6

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7793. Min Act: -0.1700

Data Index: 3616677 (Open Web Text)

Max Activating Token Index: 14

Text #7

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7331. Min Act: -0.1700

Data Index: 3988665 (Open Web Text)

Max Activating Token Index: 641

Text #8

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7486. Min Act: -0.1700

Data Index: 6535571 (Open Web Text)

Max Activating Token Index: 39

Text #9

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7396. Min Act: -0.1700

Data Index: 1169912 (Open Web Text)

Max Activating Token Index: 410

Text #10

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7085. Min Act: -0.1700

Data Index: 664783 (Open Web Text)

Max Activating Token Index: 554

Text #11

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7060. Min Act: -0.1700

Data Index: 666979 (Open Web Text)

Max Activating Token Index: 550

Text #12

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.6971. Min Act: -0.1700

Data Index: 3874597 (Open Web Text)

Max Activating Token Index: 66

Text #13

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7482. Min Act: -0.1700

Data Index: 7732238 (Open Web Text)

Max Activating Token Index: 167

Text #14

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.6889. Min Act: -0.1700

Data Index: 1164773 (Open Web Text)

Max Activating Token Index: 328

Text #15

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7109. Min Act: -0.1700

Data Index: 1938328 (Open Web Text)

Max Activating Token Index: 97

Text #16

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.7160. Min Act: -0.1700

Data Index: 2874211 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.6687. Min Act: -0.1700

Data Index: 5680031 (Open Web Text)

Max Activating Token Index: 726

Text #18

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.6820. Min Act: -0.1700

Data Index: 3614994 (Open Web Text)

Max Activating Token Index: 28

Text #19

Max Range: 2.8549. Min Range: -2.8549

Max Act: 2.6710. Min Act: -0.1700

Data Index: 4405318 (Open Web Text)

Max Activating Token Index: 101