Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2343 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0438. Min Range: -4.0438

Max Act: 4.0438. Min Act: -0.1700

Data Index: 8662337 (Open Web Text)

Max Activating Token Index: 633

Text #1

Max Range: 4.0438. Min Range: -4.0438

Max Act: 4.0258. Min Act: -0.1700

Data Index: 4966416 (Open Web Text)

Max Activating Token Index: 228

Text #2

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.9887. Min Act: -0.1700

Data Index: 4724981 (Open Web Text)

Max Activating Token Index: 480

Text #3

Max Range: 4.0438. Min Range: -4.0438

Max Act: 4.0036. Min Act: -0.1700

Data Index: 966279 (Open Web Text)

Max Activating Token Index: 666

Text #4

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.9914. Min Act: -0.1700

Data Index: 6488243 (Open Web Text)

Max Activating Token Index: 126

Text #5

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.9860. Min Act: -0.1700

Data Index: 6730065 (Open Web Text)

Max Activating Token Index: 440

Text #6

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.9550. Min Act: -0.1700

Data Index: 8347649 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.9550. Min Act: -0.1700

Data Index: 4602807 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8590. Min Act: -0.1700

Data Index: 3291864 (Open Web Text)

Max Activating Token Index: 62

Text #9

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8794. Min Act: -0.1700

Data Index: 3885506 (Open Web Text)

Max Activating Token Index: 803

Text #10

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8520. Min Act: -0.1700

Data Index: 6212784 (Open Web Text)

Max Activating Token Index: 147

Text #11

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8398. Min Act: -0.1700

Data Index: 5973164 (Open Web Text)

Max Activating Token Index: 792

Text #12

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8403. Min Act: -0.1700

Data Index: 7538794 (Open Web Text)

Max Activating Token Index: 320

Text #13

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8458. Min Act: -0.1700

Data Index: 4765603 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8066. Min Act: -0.1700

Data Index: 5252381 (Open Web Text)

Max Activating Token Index: 421

Text #15

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8068. Min Act: -0.1700

Data Index: 1596035 (Open Web Text)

Max Activating Token Index: 197

Text #16

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.8076. Min Act: -0.1700

Data Index: 5173383 (Open Web Text)

Max Activating Token Index: 545

Text #17

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.7599. Min Act: -0.1700

Data Index: 432934 (Open Web Text)

Max Activating Token Index: 748

Text #18

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.7650. Min Act: -0.1700

Data Index: 6491380 (Open Web Text)

Max Activating Token Index: 109

Text #19

Max Range: 4.0438. Min Range: -4.0438

Max Act: 3.7810. Min Act: -0.1700

Data Index: 3186298 (Open Web Text)

Max Activating Token Index: 570