Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1999 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.9453. Min Act: -0.1700

Data Index: 674977 (Open Web Text)

Max Activating Token Index: 491

Text #1

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.7767. Min Act: -0.1700

Data Index: 4064999 (Open Web Text)

Max Activating Token Index: 139

Text #2

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.7354. Min Act: -0.1700

Data Index: 2711633 (Open Web Text)

Max Activating Token Index: 874

Text #3

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.7277. Min Act: -0.1700

Data Index: 3313201 (Open Web Text)

Max Activating Token Index: 272

Text #4

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.7563. Min Act: -0.1700

Data Index: 2718826 (Open Web Text)

Max Activating Token Index: 674

Text #5

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.7342. Min Act: -0.1700

Data Index: 6136669 (Open Web Text)

Max Activating Token Index: 71

Text #6

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6989. Min Act: -0.1700

Data Index: 5145242 (Open Web Text)

Max Activating Token Index: 264

Text #7

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6875. Min Act: -0.1700

Data Index: 3823144 (Open Web Text)

Max Activating Token Index: 95

Text #8

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6965. Min Act: -0.1700

Data Index: 230405 (Open Web Text)

Max Activating Token Index: 396

Text #9

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6905. Min Act: -0.1700

Data Index: 5483116 (Open Web Text)

Max Activating Token Index: 14

Text #10

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6468. Min Act: -0.1700

Data Index: 4994820 (Open Web Text)

Max Activating Token Index: 277

Text #11

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6701. Min Act: -0.1700

Data Index: 2669024 (Open Web Text)

Max Activating Token Index: 114

Text #12

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6551. Min Act: -0.1700

Data Index: 3780508 (Open Web Text)

Max Activating Token Index: 639

Text #13

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6575. Min Act: -0.1700

Data Index: 1972046 (Open Web Text)

Max Activating Token Index: 846

Text #14

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6656. Min Act: -0.1700

Data Index: 5450606 (Open Web Text)

Max Activating Token Index: 522

Text #15

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6760. Min Act: -0.1700

Data Index: 4848820 (Open Web Text)

Max Activating Token Index: 837

Text #16

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6447. Min Act: -0.1700

Data Index: 5579679 (Open Web Text)

Max Activating Token Index: 984

Text #17

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6558. Min Act: -0.1700

Data Index: 5598926 (Open Web Text)

Max Activating Token Index: 719

Text #18

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6747. Min Act: -0.1700

Data Index: 5964773 (Open Web Text)

Max Activating Token Index: 309

Text #19

Max Range: 3.9453. Min Range: -3.9453

Max Act: 3.6506. Min Act: -0.1700

Data Index: 2621541 (Open Web Text)

Max Activating Token Index: 188