Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2104 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0430. Min Range: -5.0430

Max Act: 5.0430. Min Act: -0.1700

Data Index: 7268710 (Open Web Text)

Max Activating Token Index: 427

Text #1

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.8623. Min Act: -0.1700

Data Index: 2790859 (Open Web Text)

Max Activating Token Index: 587

Text #2

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.8880. Min Act: -0.1700

Data Index: 6778463 (Open Web Text)

Max Activating Token Index: 483

Text #3

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.8108. Min Act: -0.1700

Data Index: 7131985 (Open Web Text)

Max Activating Token Index: 794

Text #4

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7541. Min Act: -0.1700

Data Index: 6241343 (Open Web Text)

Max Activating Token Index: 865

Text #5

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7598. Min Act: -0.1700

Data Index: 516074 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7320. Min Act: -0.1700

Data Index: 8270149 (Open Web Text)

Max Activating Token Index: 876

Text #7

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7758. Min Act: -0.1700

Data Index: 1065356 (Open Web Text)

Max Activating Token Index: 511

Text #8

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7638. Min Act: -0.1700

Data Index: 8668531 (Open Web Text)

Max Activating Token Index: 339

Text #9

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7516. Min Act: -0.1700

Data Index: 1997752 (Open Web Text)

Max Activating Token Index: 798

Text #10

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7770. Min Act: -0.1700

Data Index: 1871817 (Open Web Text)

Max Activating Token Index: 609

Text #11

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7349. Min Act: -0.1700

Data Index: 5443614 (Open Web Text)

Max Activating Token Index: 962

Text #12

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7500. Min Act: -0.1700

Data Index: 3625357 (Open Web Text)

Max Activating Token Index: 618

Text #13

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7051. Min Act: -0.1700

Data Index: 2831797 (Open Web Text)

Max Activating Token Index: 626

Text #14

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7538. Min Act: -0.1700

Data Index: 3621459 (Open Web Text)

Max Activating Token Index: 943

Text #15

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7214. Min Act: -0.1700

Data Index: 226900 (Open Web Text)

Max Activating Token Index: 639

Text #16

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7514. Min Act: -0.1700

Data Index: 4448330 (Open Web Text)

Max Activating Token Index: 132

Text #17

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7028. Min Act: -0.1700

Data Index: 384651 (Open Web Text)

Max Activating Token Index: 231

Text #18

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7053. Min Act: -0.1700

Data Index: 1556683 (Open Web Text)

Max Activating Token Index: 543

Text #19

Max Range: 5.0430. Min Range: -5.0430

Max Act: 4.7195. Min Act: -0.1700

Data Index: 6546733 (Open Web Text)

Max Activating Token Index: 325