Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2044 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.7525. Min Act: -0.1700

Data Index: 3669857 (Open Web Text)

Max Activating Token Index: 202

Text #1

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.6151. Min Act: -0.1700

Data Index: 606112 (Open Web Text)

Max Activating Token Index: 725

Text #2

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.6226. Min Act: -0.1700

Data Index: 2463357 (Open Web Text)

Max Activating Token Index: 630

Text #3

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.5831. Min Act: -0.1700

Data Index: 1428107 (Open Web Text)

Max Activating Token Index: 420

Text #4

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.5309. Min Act: -0.1700

Data Index: 2677004 (Open Web Text)

Max Activating Token Index: 818

Text #5

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.6311. Min Act: -0.1700

Data Index: 2006046 (Open Web Text)

Max Activating Token Index: 1007

Text #6

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.5690. Min Act: -0.1700

Data Index: 4107289 (Open Web Text)

Max Activating Token Index: 326

Text #7

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4866. Min Act: -0.1700

Data Index: 6051264 (Open Web Text)

Max Activating Token Index: 36

Text #8

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.5296. Min Act: -0.1700

Data Index: 2011552 (Open Web Text)

Max Activating Token Index: 940

Text #9

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.5093. Min Act: -0.1700

Data Index: 6446515 (Open Web Text)

Max Activating Token Index: 500

Text #10

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4991. Min Act: -0.1700

Data Index: 8475574 (Open Web Text)

Max Activating Token Index: 948

Text #11

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4651. Min Act: -0.1700

Data Index: 6118647 (Open Web Text)

Max Activating Token Index: 728

Text #12

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4864. Min Act: -0.1700

Data Index: 7727964 (Open Web Text)

Max Activating Token Index: 608

Text #13

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4305. Min Act: -0.1700

Data Index: 4310178 (Open Web Text)

Max Activating Token Index: 934

Text #14

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4462. Min Act: -0.1700

Data Index: 6430892 (Open Web Text)

Max Activating Token Index: 501

Text #15

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4235. Min Act: -0.1700

Data Index: 5460974 (Open Web Text)

Max Activating Token Index: 104

Text #16

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4081. Min Act: -0.1700

Data Index: 2098341 (Open Web Text)

Max Activating Token Index: 196

Text #17

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4117. Min Act: -0.1700

Data Index: 4107290 (Open Web Text)

Max Activating Token Index: 51

Text #18

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.3886. Min Act: -0.1700

Data Index: 6029224 (Open Web Text)

Max Activating Token Index: 663

Text #19

Max Range: 4.7525. Min Range: -4.7525

Max Act: 4.4054. Min Act: -0.1700

Data Index: 6803480 (Open Web Text)

Max Activating Token Index: 851