Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2504 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3742. Min Act: -0.1700

Data Index: 5449493 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3352. Min Act: -0.1700

Data Index: 475286 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3394. Min Act: -0.1700

Data Index: 5243513 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3404. Min Act: -0.1700

Data Index: 1735286 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3177. Min Act: -0.1700

Data Index: 6163053 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3247. Min Act: -0.1700

Data Index: 2819295 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3125. Min Act: -0.1700

Data Index: 3886637 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3339. Min Act: -0.1700

Data Index: 3716747 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3251. Min Act: -0.1700

Data Index: 8620112 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2766. Min Act: -0.1700

Data Index: 3693992 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.3004. Min Act: -0.1700

Data Index: 6439982 (Open Web Text)

Max Activating Token Index: 846

Text #11

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2555. Min Act: -0.1700

Data Index: 5460344 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2507. Min Act: -0.1700

Data Index: 3723732 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2625. Min Act: -0.1700

Data Index: 2521090 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2727. Min Act: -0.1700

Data Index: 6511644 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2523. Min Act: -0.1700

Data Index: 4499873 (Open Web Text)

Max Activating Token Index: 39

Text #16

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2462. Min Act: -0.1700

Data Index: 6957424 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2533. Min Act: -0.1700

Data Index: 4614084 (Open Web Text)

Max Activating Token Index: 8

Text #18

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2523. Min Act: -0.1700

Data Index: 5239486 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 3.3742. Min Range: -3.3742

Max Act: 3.2671. Min Act: -0.1700

Data Index: 986409 (Open Web Text)

Max Activating Token Index: 5