Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 451 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.7228. Min Act: -0.1700

Data Index: 2495874 (Open Web Text)

Max Activating Token Index: 893

Text #1

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.5138. Min Act: -0.1700

Data Index: 2114971 (Open Web Text)

Max Activating Token Index: 545

Text #2

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.5319. Min Act: -0.1700

Data Index: 3568197 (Open Web Text)

Max Activating Token Index: 701

Text #3

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.5371. Min Act: -0.1700

Data Index: 5052783 (Open Web Text)

Max Activating Token Index: 866

Text #4

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.5351. Min Act: -0.1700

Data Index: 5128375 (Open Web Text)

Max Activating Token Index: 179

Text #5

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4688. Min Act: -0.1700

Data Index: 2801392 (Open Web Text)

Max Activating Token Index: 736

Text #6

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4932. Min Act: -0.1700

Data Index: 4662941 (Open Web Text)

Max Activating Token Index: 671

Text #7

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4815. Min Act: -0.1700

Data Index: 4146792 (Open Web Text)

Max Activating Token Index: 737

Text #8

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4666. Min Act: -0.1700

Data Index: 1328592 (Open Web Text)

Max Activating Token Index: 988

Text #9

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4590. Min Act: -0.1700

Data Index: 8812030 (Open Web Text)

Max Activating Token Index: 230

Text #10

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.5092. Min Act: -0.1700

Data Index: 7501984 (Open Web Text)

Max Activating Token Index: 898

Text #11

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4294. Min Act: -0.1700

Data Index: 428886 (Open Web Text)

Max Activating Token Index: 625

Text #12

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4713. Min Act: -0.1700

Data Index: 685951 (Open Web Text)

Max Activating Token Index: 868

Text #13

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4375. Min Act: -0.1700

Data Index: 7983490 (Open Web Text)

Max Activating Token Index: 876

Text #14

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4217. Min Act: -0.1700

Data Index: 2053054 (Open Web Text)

Max Activating Token Index: 482

Text #15

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4218. Min Act: -0.1700

Data Index: 8267409 (Open Web Text)

Max Activating Token Index: 834

Text #16

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4089. Min Act: -0.1700

Data Index: 8769952 (Open Web Text)

Max Activating Token Index: 659

Text #17

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4467. Min Act: -0.1700

Data Index: 5615814 (Open Web Text)

Max Activating Token Index: 992

Text #18

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.4288. Min Act: -0.1700

Data Index: 1286186 (Open Web Text)

Max Activating Token Index: 934

Text #19

Max Range: 3.7228. Min Range: -3.7228

Max Act: 3.3986. Min Act: -0.1700

Data Index: 5903502 (Open Web Text)

Max Activating Token Index: 788