Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2204 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.5100. Min Act: -0.1700

Data Index: 3799595 (Open Web Text)

Max Activating Token Index: 703

Text #1

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.4427. Min Act: -0.1700

Data Index: 4655344 (Open Web Text)

Max Activating Token Index: 691

Text #2

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.4679. Min Act: -0.1700

Data Index: 1021405 (Open Web Text)

Max Activating Token Index: 65

Text #3

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.4391. Min Act: -0.1700

Data Index: 5199336 (Open Web Text)

Max Activating Token Index: 625

Text #4

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3457. Min Act: -0.1700

Data Index: 5400909 (Open Web Text)

Max Activating Token Index: 667

Text #5

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3321. Min Act: -0.1700

Data Index: 2403484 (Open Web Text)

Max Activating Token Index: 508

Text #6

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3545. Min Act: -0.1700

Data Index: 813843 (Open Web Text)

Max Activating Token Index: 878

Text #7

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3510. Min Act: -0.1700

Data Index: 7525101 (Open Web Text)

Max Activating Token Index: 566

Text #8

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3081. Min Act: -0.1700

Data Index: 350500 (Open Web Text)

Max Activating Token Index: 368

Text #9

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3410. Min Act: -0.1700

Data Index: 7701506 (Open Web Text)

Max Activating Token Index: 893

Text #10

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3391. Min Act: -0.1700

Data Index: 8729249 (Open Web Text)

Max Activating Token Index: 350

Text #11

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3059. Min Act: -0.1700

Data Index: 233268 (Open Web Text)

Max Activating Token Index: 137

Text #12

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.3009. Min Act: -0.1700

Data Index: 1453362 (Open Web Text)

Max Activating Token Index: 317

Text #13

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2924. Min Act: -0.1700

Data Index: 1229696 (Open Web Text)

Max Activating Token Index: 659

Text #14

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2981. Min Act: -0.1700

Data Index: 3249006 (Open Web Text)

Max Activating Token Index: 241

Text #15

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2441. Min Act: -0.1700

Data Index: 586008 (Open Web Text)

Max Activating Token Index: 393

Text #16

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2661. Min Act: -0.1700

Data Index: 4694465 (Open Web Text)

Max Activating Token Index: 469

Text #17

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2676. Min Act: -0.1700

Data Index: 2752655 (Open Web Text)

Max Activating Token Index: 362

Text #18

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2658. Min Act: -0.1700

Data Index: 5175499 (Open Web Text)

Max Activating Token Index: 162

Text #19

Max Range: 4.5100. Min Range: -4.5100

Max Act: 4.2813. Min Act: -0.1700

Data Index: 657795 (Open Web Text)

Max Activating Token Index: 477