Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1400 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.8430. Min Act: -0.1700

Data Index: 973501 (Open Web Text)

Max Activating Token Index: 989

Text #1

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.8277. Min Act: -0.1700

Data Index: 6332617 (Open Web Text)

Max Activating Token Index: 13

Text #2

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.8144. Min Act: -0.1700

Data Index: 5759741 (Open Web Text)

Max Activating Token Index: 377

Text #3

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7904. Min Act: -0.1700

Data Index: 3059929 (Open Web Text)

Max Activating Token Index: 657

Text #4

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7540. Min Act: -0.1700

Data Index: 2501585 (Open Web Text)

Max Activating Token Index: 565

Text #5

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7880. Min Act: -0.1700

Data Index: 4735159 (Open Web Text)

Max Activating Token Index: 909

Text #6

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7719. Min Act: -0.1700

Data Index: 5124500 (Open Web Text)

Max Activating Token Index: 350

Text #7

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7826. Min Act: -0.1700

Data Index: 4242722 (Open Web Text)

Max Activating Token Index: 634

Text #8

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7420. Min Act: -0.1700

Data Index: 8258700 (Open Web Text)

Max Activating Token Index: 895

Text #9

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7470. Min Act: -0.1700

Data Index: 180472 (Open Web Text)

Max Activating Token Index: 482

Text #10

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7226. Min Act: -0.1700

Data Index: 739977 (Open Web Text)

Max Activating Token Index: 114

Text #11

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7359. Min Act: -0.1700

Data Index: 6037698 (Open Web Text)

Max Activating Token Index: 599

Text #12

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7438. Min Act: -0.1700

Data Index: 8101260 (Open Web Text)

Max Activating Token Index: 994

Text #13

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7588. Min Act: -0.1700

Data Index: 2869143 (Open Web Text)

Max Activating Token Index: 923

Text #14

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7341. Min Act: -0.1700

Data Index: 1092998 (Open Web Text)

Max Activating Token Index: 923

Text #15

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7174. Min Act: -0.1700

Data Index: 5165901 (Open Web Text)

Max Activating Token Index: 377

Text #16

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7296. Min Act: -0.1700

Data Index: 862296 (Open Web Text)

Max Activating Token Index: 504

Text #17

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7202. Min Act: -0.1700

Data Index: 1910519 (Open Web Text)

Max Activating Token Index: 949

Text #18

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7085. Min Act: -0.1700

Data Index: 1426762 (Open Web Text)

Max Activating Token Index: 214

Text #19

Max Range: 2.8430. Min Range: -2.8430

Max Act: 2.7209. Min Act: -0.1700

Data Index: 1010435 (Open Web Text)

Max Activating Token Index: 98