Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 571 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7764. Min Act: -0.1700

Data Index: 3607369 (Open Web Text)

Max Activating Token Index: 957

Text #1

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7431. Min Act: -0.1700

Data Index: 6446836 (Open Web Text)

Max Activating Token Index: 667

Text #2

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7282. Min Act: -0.1700

Data Index: 7310930 (Open Web Text)

Max Activating Token Index: 434

Text #3

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7336. Min Act: -0.1700

Data Index: 7420 (Open Web Text)

Max Activating Token Index: 695

Text #4

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7374. Min Act: -0.1700

Data Index: 1268343 (Open Web Text)

Max Activating Token Index: 954

Text #5

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7274. Min Act: -0.1700

Data Index: 622880 (Open Web Text)

Max Activating Token Index: 476

Text #6

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7321. Min Act: -0.1700

Data Index: 6152694 (Open Web Text)

Max Activating Token Index: 220

Text #7

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7053. Min Act: -0.1700

Data Index: 3666489 (Open Web Text)

Max Activating Token Index: 557

Text #8

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.7040. Min Act: -0.1700

Data Index: 534981 (Open Web Text)

Max Activating Token Index: 858

Text #9

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6908. Min Act: -0.1700

Data Index: 4048638 (Open Web Text)

Max Activating Token Index: 1001

Text #10

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6988. Min Act: -0.1700

Data Index: 964471 (Open Web Text)

Max Activating Token Index: 789

Text #11

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6895. Min Act: -0.1700

Data Index: 1859527 (Open Web Text)

Max Activating Token Index: 893

Text #12

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6818. Min Act: -0.1700

Data Index: 7248381 (Open Web Text)

Max Activating Token Index: 137

Text #13

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6888. Min Act: -0.1700

Data Index: 3541689 (Open Web Text)

Max Activating Token Index: 713

Text #14

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6726. Min Act: -0.1700

Data Index: 6858149 (Open Web Text)

Max Activating Token Index: 174

Text #15

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6765. Min Act: -0.1700

Data Index: 7436057 (Open Web Text)

Max Activating Token Index: 615

Text #16

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6828. Min Act: -0.1700

Data Index: 1399220 (Open Web Text)

Max Activating Token Index: 505

Text #17

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6874. Min Act: -0.1700

Data Index: 5249032 (Open Web Text)

Max Activating Token Index: 734

Text #18

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6732. Min Act: -0.1700

Data Index: 1004373 (Open Web Text)

Max Activating Token Index: 899

Text #19

Max Range: 1.7764. Min Range: -1.7764

Max Act: 1.6744. Min Act: -0.1700

Data Index: 7572565 (Open Web Text)

Max Activating Token Index: 143