Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2163 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.4499. Min Act: -0.1700

Data Index: 2871848 (Open Web Text)

Max Activating Token Index: 784

Text #1

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.3326. Min Act: -0.1700

Data Index: 445625 (Open Web Text)

Max Activating Token Index: 446

Text #2

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.3418. Min Act: -0.1700

Data Index: 2416178 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2776. Min Act: -0.1700

Data Index: 1461307 (Open Web Text)

Max Activating Token Index: 301

Text #4

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.3059. Min Act: -0.1700

Data Index: 7845553 (Open Web Text)

Max Activating Token Index: 656

Text #5

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2549. Min Act: -0.1700

Data Index: 3715654 (Open Web Text)

Max Activating Token Index: 537

Text #6

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2408. Min Act: -0.1700

Data Index: 2781723 (Open Web Text)

Max Activating Token Index: 603

Text #7

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2624. Min Act: -0.1700

Data Index: 5189895 (Open Web Text)

Max Activating Token Index: 450

Text #8

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2721. Min Act: -0.1700

Data Index: 2678427 (Open Web Text)

Max Activating Token Index: 823

Text #9

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2510. Min Act: -0.1700

Data Index: 3701762 (Open Web Text)

Max Activating Token Index: 562

Text #10

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2587. Min Act: -0.1700

Data Index: 8781732 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2639. Min Act: -0.1700

Data Index: 4286247 (Open Web Text)

Max Activating Token Index: 608

Text #12

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2301. Min Act: -0.1700

Data Index: 8765831 (Open Web Text)

Max Activating Token Index: 439

Text #13

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2583. Min Act: -0.1700

Data Index: 3820805 (Open Web Text)

Max Activating Token Index: 749

Text #14

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2326. Min Act: -0.1700

Data Index: 3347528 (Open Web Text)

Max Activating Token Index: 489

Text #15

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2284. Min Act: -0.1700

Data Index: 7619957 (Open Web Text)

Max Activating Token Index: 544

Text #16

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2228. Min Act: -0.1700

Data Index: 6164962 (Open Web Text)

Max Activating Token Index: 652

Text #17

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2193. Min Act: -0.1700

Data Index: 5311490 (Open Web Text)

Max Activating Token Index: 982

Text #18

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2175. Min Act: -0.1700

Data Index: 59547 (Open Web Text)

Max Activating Token Index: 436

Text #19

Max Range: 4.4499. Min Range: -4.4499

Max Act: 4.2359. Min Act: -0.1700

Data Index: 5188714 (Open Web Text)

Max Activating Token Index: 438