Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 152 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.4501. Min Act: -0.1700

Data Index: 836494 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.4209. Min Act: -0.1700

Data Index: 2232341 (Open Web Text)

Max Activating Token Index: 27

Text #2

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3823. Min Act: -0.1700

Data Index: 7582109 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3833. Min Act: -0.1700

Data Index: 1164908 (Open Web Text)

Max Activating Token Index: 933

Text #4

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3638. Min Act: -0.1700

Data Index: 6714106 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3622. Min Act: -0.1700

Data Index: 1617931 (Open Web Text)

Max Activating Token Index: 12

Text #6

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3677. Min Act: -0.1700

Data Index: 8705328 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3601. Min Act: -0.1700

Data Index: 4750706 (Open Web Text)

Max Activating Token Index: 13

Text #8

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3697. Min Act: -0.1700

Data Index: 7733263 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3601. Min Act: -0.1700

Data Index: 1170527 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3437. Min Act: -0.1700

Data Index: 4810729 (Open Web Text)

Max Activating Token Index: 406

Text #11

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3443. Min Act: -0.1700

Data Index: 5637551 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3257. Min Act: -0.1700

Data Index: 5985524 (Open Web Text)

Max Activating Token Index: 884

Text #13

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3491. Min Act: -0.1700

Data Index: 2243646 (Open Web Text)

Max Activating Token Index: 373

Text #14

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3431. Min Act: -0.1700

Data Index: 2279991 (Open Web Text)

Max Activating Token Index: 78

Text #15

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3456. Min Act: -0.1700

Data Index: 2410079 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3364. Min Act: -0.1700

Data Index: 955508 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3596. Min Act: -0.1700

Data Index: 2785933 (Open Web Text)

Max Activating Token Index: 841

Text #18

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3439. Min Act: -0.1700

Data Index: 6162926 (Open Web Text)

Max Activating Token Index: 791

Text #19

Max Range: 3.4501. Min Range: -3.4501

Max Act: 3.3365. Min Act: -0.1700

Data Index: 6274025 (Open Web Text)

Max Activating Token Index: 11