Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1728 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.9208. Min Act: -0.1700

Data Index: 3810420 (Open Web Text)

Max Activating Token Index: 480

Text #1

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.8471. Min Act: -0.1700

Data Index: 328357 (Open Web Text)

Max Activating Token Index: 711

Text #2

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7997. Min Act: -0.1700

Data Index: 7033027 (Open Web Text)

Max Activating Token Index: 939

Text #3

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.8201. Min Act: -0.1700

Data Index: 7992897 (Open Web Text)

Max Activating Token Index: 779

Text #4

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.8085. Min Act: -0.1700

Data Index: 7244682 (Open Web Text)

Max Activating Token Index: 725

Text #5

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.8012. Min Act: -0.1700

Data Index: 2390047 (Open Web Text)

Max Activating Token Index: 676

Text #6

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7630. Min Act: -0.1700

Data Index: 215021 (Open Web Text)

Max Activating Token Index: 195

Text #7

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7979. Min Act: -0.1700

Data Index: 5846981 (Open Web Text)

Max Activating Token Index: 444

Text #8

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7562. Min Act: -0.1700

Data Index: 6084632 (Open Web Text)

Max Activating Token Index: 828

Text #9

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7671. Min Act: -0.1700

Data Index: 6180735 (Open Web Text)

Max Activating Token Index: 828

Text #10

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.6999. Min Act: -0.1700

Data Index: 5013189 (Open Web Text)

Max Activating Token Index: 478

Text #11

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7496. Min Act: -0.1700

Data Index: 19415 (Open Web Text)

Max Activating Token Index: 603

Text #12

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7150. Min Act: -0.1700

Data Index: 6545133 (Open Web Text)

Max Activating Token Index: 408

Text #13

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7180. Min Act: -0.1700

Data Index: 7999357 (Open Web Text)

Max Activating Token Index: 544

Text #14

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7262. Min Act: -0.1700

Data Index: 5305951 (Open Web Text)

Max Activating Token Index: 712

Text #15

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.7503. Min Act: -0.1700

Data Index: 7769122 (Open Web Text)

Max Activating Token Index: 60

Text #16

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.6790. Min Act: -0.1700

Data Index: 3305868 (Open Web Text)

Max Activating Token Index: 347

Text #17

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.6876. Min Act: -0.1700

Data Index: 1049861 (Open Web Text)

Max Activating Token Index: 151

Text #18

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.6631. Min Act: -0.1700

Data Index: 4863128 (Open Web Text)

Max Activating Token Index: 449

Text #19

Max Range: 4.9208. Min Range: -4.9208

Max Act: 4.6886. Min Act: -0.1700

Data Index: 3344772 (Open Web Text)

Max Activating Token Index: 806