Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1248 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.7658. Min Act: -0.1700

Data Index: 5330513 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.7357. Min Act: -0.1700

Data Index: 1728440 (Open Web Text)

Max Activating Token Index: 14

Text #2

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.7402. Min Act: -0.1700

Data Index: 500510 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.7101. Min Act: -0.1700

Data Index: 8271379 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6546. Min Act: -0.1700

Data Index: 8070047 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6602. Min Act: -0.1700

Data Index: 7861575 (Open Web Text)

Max Activating Token Index: 11

Text #6

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6462. Min Act: -0.1700

Data Index: 108519 (Open Web Text)

Max Activating Token Index: 9

Text #7

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6282. Min Act: -0.1700

Data Index: 7584759 (Open Web Text)

Max Activating Token Index: 19

Text #8

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6062. Min Act: -0.1700

Data Index: 4992343 (Open Web Text)

Max Activating Token Index: 13

Text #9

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.6167. Min Act: -0.1700

Data Index: 6492661 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.5775. Min Act: -0.1700

Data Index: 3428733 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.5302. Min Act: -0.1700

Data Index: 8689469 (Open Web Text)

Max Activating Token Index: 16

Text #12

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.5365. Min Act: -0.1700

Data Index: 6291720 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4946. Min Act: -0.1700

Data Index: 7530213 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.5002. Min Act: -0.1700

Data Index: 5285536 (Open Web Text)

Max Activating Token Index: 10

Text #15

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4828. Min Act: -0.1700

Data Index: 7513346 (Open Web Text)

Max Activating Token Index: 19

Text #16

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4624. Min Act: -0.1700

Data Index: 2268815 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4747. Min Act: -0.1700

Data Index: 5650712 (Open Web Text)

Max Activating Token Index: 783

Text #18

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4489. Min Act: -0.1700

Data Index: 6627586 (Open Web Text)

Max Activating Token Index: 767

Text #19

Max Range: 4.7658. Min Range: -4.7658

Max Act: 4.4681. Min Act: -0.1700

Data Index: 7386596 (Open Web Text)

Max Activating Token Index: 11