Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 45 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.7465. Min Act: -0.1700

Data Index: 6926782 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.7044. Min Act: -0.1700

Data Index: 224801 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.7063. Min Act: -0.1700

Data Index: 3068554 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6872. Min Act: -0.1700

Data Index: 3182221 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6483. Min Act: -0.1700

Data Index: 3466558 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6418. Min Act: -0.1700

Data Index: 4512301 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6493. Min Act: -0.1700

Data Index: 8166622 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6605. Min Act: -0.1700

Data Index: 5674939 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6383. Min Act: -0.1700

Data Index: 1908980 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6335. Min Act: -0.1700

Data Index: 339408 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6643. Min Act: -0.1700

Data Index: 5633394 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6514. Min Act: -0.1700

Data Index: 4820396 (Open Web Text)

Max Activating Token Index: 2

Text #12

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6294. Min Act: -0.1700

Data Index: 7358228 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6359. Min Act: -0.1700

Data Index: 814757 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6072. Min Act: -0.1700

Data Index: 7423060 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6108. Min Act: -0.1700

Data Index: 8589430 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6178. Min Act: -0.1700

Data Index: 4398107 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6248. Min Act: -0.1700

Data Index: 7136197 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.6228. Min Act: -0.1700

Data Index: 2505528 (Open Web Text)

Max Activating Token Index: 840

Text #19

Max Range: 3.7465. Min Range: -3.7465

Max Act: 3.5826. Min Act: -0.1700

Data Index: 5940747 (Open Web Text)

Max Activating Token Index: 5