Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 946 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3031. Min Range: -5.3031

Max Act: 5.3031. Min Act: -0.1700

Data Index: 5943948 (Open Web Text)

Max Activating Token Index: 951

Text #1

Max Range: 5.3031. Min Range: -5.3031

Max Act: 5.0959. Min Act: -0.1700

Data Index: 769410 (Open Web Text)

Max Activating Token Index: 303

Text #2

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8928. Min Act: -0.1700

Data Index: 326734 (Open Web Text)

Max Activating Token Index: 880

Text #3

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.9509. Min Act: -0.1700

Data Index: 8584525 (Open Web Text)

Max Activating Token Index: 370

Text #4

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8701. Min Act: -0.1700

Data Index: 2959408 (Open Web Text)

Max Activating Token Index: 708

Text #5

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8418. Min Act: -0.1700

Data Index: 1459687 (Open Web Text)

Max Activating Token Index: 1015

Text #6

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8506. Min Act: -0.1700

Data Index: 216671 (Open Web Text)

Max Activating Token Index: 618

Text #7

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8388. Min Act: -0.1700

Data Index: 391555 (Open Web Text)

Max Activating Token Index: 538

Text #8

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7476. Min Act: -0.1700

Data Index: 7316376 (Open Web Text)

Max Activating Token Index: 478

Text #9

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.8190. Min Act: -0.1700

Data Index: 6413170 (Open Web Text)

Max Activating Token Index: 491

Text #10

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7652. Min Act: -0.1700

Data Index: 5298803 (Open Web Text)

Max Activating Token Index: 688

Text #11

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7771. Min Act: -0.1700

Data Index: 6846666 (Open Web Text)

Max Activating Token Index: 741

Text #12

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7839. Min Act: -0.1700

Data Index: 6829307 (Open Web Text)

Max Activating Token Index: 275

Text #13

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7624. Min Act: -0.1700

Data Index: 5443250 (Open Web Text)

Max Activating Token Index: 499

Text #14

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7095. Min Act: -0.1700

Data Index: 1921139 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7567. Min Act: -0.1700

Data Index: 8600315 (Open Web Text)

Max Activating Token Index: 327

Text #16

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7775. Min Act: -0.1700

Data Index: 2671743 (Open Web Text)

Max Activating Token Index: 897

Text #17

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7120. Min Act: -0.1700

Data Index: 3195537 (Open Web Text)

Max Activating Token Index: 688

Text #18

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7203. Min Act: -0.1700

Data Index: 6609892 (Open Web Text)

Max Activating Token Index: 453

Text #19

Max Range: 5.3031. Min Range: -5.3031

Max Act: 4.7825. Min Act: -0.1700

Data Index: 7282914 (Open Web Text)

Max Activating Token Index: 910