Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1467 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1072. Min Range: -3.1072

Max Act: 3.1072. Min Act: -0.1700

Data Index: 4498288 (Open Web Text)

Max Activating Token Index: 211

Text #1

Max Range: 3.1072. Min Range: -3.1072

Max Act: 3.0194. Min Act: -0.1700

Data Index: 3493351 (Open Web Text)

Max Activating Token Index: 771

Text #2

Max Range: 3.1072. Min Range: -3.1072

Max Act: 3.0249. Min Act: -0.1700

Data Index: 1171158 (Open Web Text)

Max Activating Token Index: 553

Text #3

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8830. Min Act: -0.1700

Data Index: 4914071 (Open Web Text)

Max Activating Token Index: 775

Text #4

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8859. Min Act: -0.1700

Data Index: 1037295 (Open Web Text)

Max Activating Token Index: 279

Text #5

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8977. Min Act: -0.1700

Data Index: 4679482 (Open Web Text)

Max Activating Token Index: 613

Text #6

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8627. Min Act: -0.1700

Data Index: 257847 (Open Web Text)

Max Activating Token Index: 273

Text #7

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8457. Min Act: -0.1700

Data Index: 5993440 (Open Web Text)

Max Activating Token Index: 280

Text #8

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8521. Min Act: -0.1700

Data Index: 4180376 (Open Web Text)

Max Activating Token Index: 742

Text #9

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8246. Min Act: -0.1700

Data Index: 4123125 (Open Web Text)

Max Activating Token Index: 810

Text #10

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8076. Min Act: -0.1700

Data Index: 8352870 (Open Web Text)

Max Activating Token Index: 177

Text #11

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8229. Min Act: -0.1700

Data Index: 6710749 (Open Web Text)

Max Activating Token Index: 291

Text #12

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8000. Min Act: -0.1700

Data Index: 5501590 (Open Web Text)

Max Activating Token Index: 666

Text #13

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8049. Min Act: -0.1700

Data Index: 6922650 (Open Web Text)

Max Activating Token Index: 558

Text #14

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8192. Min Act: -0.1700

Data Index: 6157543 (Open Web Text)

Max Activating Token Index: 905

Text #15

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.8019. Min Act: -0.1700

Data Index: 6979000 (Open Web Text)

Max Activating Token Index: 114

Text #16

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.7971. Min Act: -0.1700

Data Index: 614544 (Open Web Text)

Max Activating Token Index: 364

Text #17

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.7745. Min Act: -0.1700

Data Index: 409562 (Open Web Text)

Max Activating Token Index: 508

Text #18

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.7931. Min Act: -0.1700

Data Index: 4704550 (Open Web Text)

Max Activating Token Index: 418

Text #19

Max Range: 3.1072. Min Range: -3.1072

Max Act: 2.7691. Min Act: -0.1700

Data Index: 1702860 (Open Web Text)

Max Activating Token Index: 930