Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1696 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.6390. Min Act: -0.1700

Data Index: 5491847 (Open Web Text)

Max Activating Token Index: 492

Text #1

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.5761. Min Act: -0.1700

Data Index: 937250 (Open Web Text)

Max Activating Token Index: 372

Text #2

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.5396. Min Act: -0.1700

Data Index: 6810553 (Open Web Text)

Max Activating Token Index: 561

Text #3

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4979. Min Act: -0.1700

Data Index: 5766076 (Open Web Text)

Max Activating Token Index: 932

Text #4

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4476. Min Act: -0.1700

Data Index: 1365177 (Open Web Text)

Max Activating Token Index: 47

Text #5

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4474. Min Act: -0.1700

Data Index: 6985806 (Open Web Text)

Max Activating Token Index: 277

Text #6

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4394. Min Act: -0.1700

Data Index: 7921526 (Open Web Text)

Max Activating Token Index: 647

Text #7

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4104. Min Act: -0.1700

Data Index: 3701350 (Open Web Text)

Max Activating Token Index: 569

Text #8

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3845. Min Act: -0.1700

Data Index: 8126995 (Open Web Text)

Max Activating Token Index: 600

Text #9

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3916. Min Act: -0.1700

Data Index: 7039438 (Open Web Text)

Max Activating Token Index: 622

Text #10

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.4174. Min Act: -0.1700

Data Index: 7453906 (Open Web Text)

Max Activating Token Index: 632

Text #11

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3818. Min Act: -0.1700

Data Index: 5352777 (Open Web Text)

Max Activating Token Index: 376

Text #12

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3764. Min Act: -0.1700

Data Index: 2712233 (Open Web Text)

Max Activating Token Index: 731

Text #13

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3400. Min Act: -0.1700

Data Index: 5363042 (Open Web Text)

Max Activating Token Index: 626

Text #14

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3412. Min Act: -0.1700

Data Index: 5025649 (Open Web Text)

Max Activating Token Index: 174

Text #15

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3354. Min Act: -0.1699

Data Index: 2265375 (Open Web Text)

Max Activating Token Index: 50

Text #16

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3326. Min Act: -0.1700

Data Index: 5751503 (Open Web Text)

Max Activating Token Index: 225

Text #17

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3068. Min Act: -0.1700

Data Index: 3658052 (Open Web Text)

Max Activating Token Index: 820

Text #18

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3012. Min Act: -0.1700

Data Index: 3469265 (Open Web Text)

Max Activating Token Index: 946

Text #19

Max Range: 3.6390. Min Range: -3.6390

Max Act: 3.3418. Min Act: -0.1700

Data Index: 381165 (Open Web Text)

Max Activating Token Index: 559