Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1991 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.8828. Min Act: -0.1700

Data Index: 3967422 (Open Web Text)

Max Activating Token Index: 587

Text #1

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7432. Min Act: -0.1700

Data Index: 4004589 (Open Web Text)

Max Activating Token Index: 222

Text #2

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7268. Min Act: -0.1700

Data Index: 5362774 (Open Web Text)

Max Activating Token Index: 146

Text #3

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7154. Min Act: -0.1700

Data Index: 8285028 (Open Web Text)

Max Activating Token Index: 351

Text #4

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7052. Min Act: -0.1700

Data Index: 8250174 (Open Web Text)

Max Activating Token Index: 100

Text #5

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7282. Min Act: -0.1700

Data Index: 7690379 (Open Web Text)

Max Activating Token Index: 123

Text #6

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.7171. Min Act: -0.1700

Data Index: 7396400 (Open Web Text)

Max Activating Token Index: 455

Text #7

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6554. Min Act: -0.1700

Data Index: 5836704 (Open Web Text)

Max Activating Token Index: 214

Text #8

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6919. Min Act: -0.1700

Data Index: 5362773 (Open Web Text)

Max Activating Token Index: 257

Text #9

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6909. Min Act: -0.1700

Data Index: 5810904 (Open Web Text)

Max Activating Token Index: 396

Text #10

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6639. Min Act: -0.1700

Data Index: 1023282 (Open Web Text)

Max Activating Token Index: 546

Text #11

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6459. Min Act: -0.1700

Data Index: 3754348 (Open Web Text)

Max Activating Token Index: 347

Text #12

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6442. Min Act: -0.1700

Data Index: 5647313 (Open Web Text)

Max Activating Token Index: 185

Text #13

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6380. Min Act: -0.1700

Data Index: 2931333 (Open Web Text)

Max Activating Token Index: 606

Text #14

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6573. Min Act: -0.1700

Data Index: 3203106 (Open Web Text)

Max Activating Token Index: 684

Text #15

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6182. Min Act: -0.1700

Data Index: 8459615 (Open Web Text)

Max Activating Token Index: 997

Text #16

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6302. Min Act: -0.1700

Data Index: 4873798 (Open Web Text)

Max Activating Token Index: 198

Text #17

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6213. Min Act: -0.1700

Data Index: 1541470 (Open Web Text)

Max Activating Token Index: 64

Text #18

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6314. Min Act: -0.1700

Data Index: 1541469 (Open Web Text)

Max Activating Token Index: 44

Text #19

Max Range: 2.8828. Min Range: -2.8828

Max Act: 2.6202. Min Act: -0.1700

Data Index: 3654351 (Open Web Text)

Max Activating Token Index: 5