Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2440 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.8506. Min Act: -0.1700

Data Index: 7259729 (Open Web Text)

Max Activating Token Index: 763

Text #1

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.7634. Min Act: -0.1700

Data Index: 4555467 (Open Web Text)

Max Activating Token Index: 459

Text #2

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.6910. Min Act: -0.1700

Data Index: 4525683 (Open Web Text)

Max Activating Token Index: 528

Text #3

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.5446. Min Act: -0.1700

Data Index: 232469 (Open Web Text)

Max Activating Token Index: 622

Text #4

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.5523. Min Act: -0.1700

Data Index: 7496612 (Open Web Text)

Max Activating Token Index: 861

Text #5

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.5381. Min Act: -0.1700

Data Index: 3949828 (Open Web Text)

Max Activating Token Index: 92

Text #6

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.4868. Min Act: -0.1700

Data Index: 6446162 (Open Web Text)

Max Activating Token Index: 616

Text #7

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.4535. Min Act: -0.1700

Data Index: 4663517 (Open Web Text)

Max Activating Token Index: 712

Text #8

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.4829. Min Act: -0.1700

Data Index: 2609938 (Open Web Text)

Max Activating Token Index: 792

Text #9

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.4145. Min Act: -0.1700

Data Index: 6717975 (Open Web Text)

Max Activating Token Index: 378

Text #10

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3873. Min Act: -0.1700

Data Index: 8460702 (Open Web Text)

Max Activating Token Index: 265

Text #11

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3686. Min Act: -0.1700

Data Index: 7000146 (Open Web Text)

Max Activating Token Index: 391

Text #12

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3474. Min Act: -0.1700

Data Index: 171220 (Open Web Text)

Max Activating Token Index: 564

Text #13

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3413. Min Act: -0.1700

Data Index: 8255294 (Open Web Text)

Max Activating Token Index: 845

Text #14

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3511. Min Act: -0.1700

Data Index: 4604063 (Open Web Text)

Max Activating Token Index: 909

Text #15

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3571. Min Act: -0.1700

Data Index: 4850565 (Open Web Text)

Max Activating Token Index: 738

Text #16

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.3030. Min Act: -0.1700

Data Index: 5841173 (Open Web Text)

Max Activating Token Index: 535

Text #17

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.2881. Min Act: -0.1700

Data Index: 3700394 (Open Web Text)

Max Activating Token Index: 58

Text #18

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.2997. Min Act: -0.1700

Data Index: 548611 (Open Web Text)

Max Activating Token Index: 697

Text #19

Max Range: 3.8506. Min Range: -3.8506

Max Act: 3.2820. Min Act: -0.1700

Data Index: 2399002 (Open Web Text)

Max Activating Token Index: 27