Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1838 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.5122. Min Act: -0.1700

Data Index: 6184386 (Open Web Text)

Max Activating Token Index: 279

Text #1

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.5057. Min Act: -0.1700

Data Index: 8224177 (Open Web Text)

Max Activating Token Index: 192

Text #2

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4719. Min Act: -0.1700

Data Index: 3171201 (Open Web Text)

Max Activating Token Index: 185

Text #3

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4451. Min Act: -0.1700

Data Index: 3733669 (Open Web Text)

Max Activating Token Index: 188

Text #4

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4489. Min Act: -0.1700

Data Index: 5790949 (Open Web Text)

Max Activating Token Index: 1020

Text #5

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4738. Min Act: -0.1700

Data Index: 1222052 (Open Web Text)

Max Activating Token Index: 98

Text #6

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4252. Min Act: -0.1700

Data Index: 7471288 (Open Web Text)

Max Activating Token Index: 814

Text #7

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.4097. Min Act: -0.1700

Data Index: 7137245 (Open Web Text)

Max Activating Token Index: 793

Text #8

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3731. Min Act: -0.1700

Data Index: 3336297 (Open Web Text)

Max Activating Token Index: 710

Text #9

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3988. Min Act: -0.1700

Data Index: 4532282 (Open Web Text)

Max Activating Token Index: 123

Text #10

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3957. Min Act: -0.1700

Data Index: 5460983 (Open Web Text)

Max Activating Token Index: 108

Text #11

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3721. Min Act: -0.1700

Data Index: 7966489 (Open Web Text)

Max Activating Token Index: 571

Text #12

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3574. Min Act: -0.1700

Data Index: 4390335 (Open Web Text)

Max Activating Token Index: 211

Text #13

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3191. Min Act: -0.1700

Data Index: 8489997 (Open Web Text)

Max Activating Token Index: 820

Text #14

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3462. Min Act: -0.1700

Data Index: 3263751 (Open Web Text)

Max Activating Token Index: 290

Text #15

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3663. Min Act: -0.1700

Data Index: 760099 (Open Web Text)

Max Activating Token Index: 783

Text #16

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.2906. Min Act: -0.1700

Data Index: 3733666 (Open Web Text)

Max Activating Token Index: 751

Text #17

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3003. Min Act: -0.1700

Data Index: 8005290 (Open Web Text)

Max Activating Token Index: 473

Text #18

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.3033. Min Act: -0.1700

Data Index: 5326183 (Open Web Text)

Max Activating Token Index: 731

Text #19

Max Range: 4.5122. Min Range: -4.5122

Max Act: 4.2964. Min Act: -0.1700

Data Index: 7615 (Open Web Text)

Max Activating Token Index: 851