Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 328 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.4485. Min Act: -0.1700

Data Index: 8450717 (Open Web Text)

Max Activating Token Index: 518

Text #1

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3776. Min Act: -0.1700

Data Index: 4544645 (Open Web Text)

Max Activating Token Index: 28

Text #2

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3857. Min Act: -0.1700

Data Index: 2707600 (Open Web Text)

Max Activating Token Index: 678

Text #3

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3596. Min Act: -0.1700

Data Index: 2484368 (Open Web Text)

Max Activating Token Index: 415

Text #4

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3120. Min Act: -0.1700

Data Index: 5641537 (Open Web Text)

Max Activating Token Index: 936

Text #5

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2828. Min Act: -0.1700

Data Index: 1343536 (Open Web Text)

Max Activating Token Index: 403

Text #6

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3408. Min Act: -0.1700

Data Index: 7731836 (Open Web Text)

Max Activating Token Index: 320

Text #7

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3363. Min Act: -0.1700

Data Index: 6527661 (Open Web Text)

Max Activating Token Index: 21

Text #8

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3161. Min Act: -0.1700

Data Index: 71774 (Open Web Text)

Max Activating Token Index: 258

Text #9

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2903. Min Act: -0.1700

Data Index: 1858426 (Open Web Text)

Max Activating Token Index: 101

Text #10

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3124. Min Act: -0.1700

Data Index: 1808484 (Open Web Text)

Max Activating Token Index: 326

Text #11

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.3075. Min Act: -0.1700

Data Index: 4495257 (Open Web Text)

Max Activating Token Index: 680

Text #12

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2567. Min Act: -0.1700

Data Index: 4246000 (Open Web Text)

Max Activating Token Index: 765

Text #13

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2844. Min Act: -0.1700

Data Index: 4179831 (Open Web Text)

Max Activating Token Index: 16

Text #14

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2470. Min Act: -0.1700

Data Index: 2296513 (Open Web Text)

Max Activating Token Index: 640

Text #15

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2855. Min Act: -0.1700

Data Index: 7510775 (Open Web Text)

Max Activating Token Index: 955

Text #16

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2389. Min Act: -0.1700

Data Index: 7123371 (Open Web Text)

Max Activating Token Index: 568

Text #17

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2561. Min Act: -0.1700

Data Index: 6871991 (Open Web Text)

Max Activating Token Index: 209

Text #18

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2755. Min Act: -0.1700

Data Index: 413493 (Open Web Text)

Max Activating Token Index: 785

Text #19

Max Range: 4.4485. Min Range: -4.4485

Max Act: 4.2635. Min Act: -0.1700

Data Index: 5824815 (Open Web Text)

Max Activating Token Index: 1019