Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1177 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.3948. Min Act: -0.1700

Data Index: 2701508 (Open Web Text)

Max Activating Token Index: 611

Text #1

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.3314. Min Act: -0.1700

Data Index: 6159815 (Open Web Text)

Max Activating Token Index: 631

Text #2

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.2082. Min Act: -0.1700

Data Index: 5601321 (Open Web Text)

Max Activating Token Index: 719

Text #3

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.1861. Min Act: -0.1700

Data Index: 2774112 (Open Web Text)

Max Activating Token Index: 380

Text #4

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.1591. Min Act: -0.1700

Data Index: 2416476 (Open Web Text)

Max Activating Token Index: 346

Text #5

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.1625. Min Act: -0.1700

Data Index: 6345716 (Open Web Text)

Max Activating Token Index: 639

Text #6

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.1355. Min Act: -0.1700

Data Index: 7400344 (Open Web Text)

Max Activating Token Index: 847

Text #7

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.1049. Min Act: -0.1700

Data Index: 2907511 (Open Web Text)

Max Activating Token Index: 111

Text #8

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0977. Min Act: -0.1700

Data Index: 2095011 (Open Web Text)

Max Activating Token Index: 640

Text #9

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0888. Min Act: -0.1700

Data Index: 2689502 (Open Web Text)

Max Activating Token Index: 79

Text #10

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0651. Min Act: -0.1700

Data Index: 1738960 (Open Web Text)

Max Activating Token Index: 178

Text #11

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0641. Min Act: -0.1700

Data Index: 6010769 (Open Web Text)

Max Activating Token Index: 522

Text #12

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0569. Min Act: -0.1700

Data Index: 6117656 (Open Web Text)

Max Activating Token Index: 243

Text #13

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0488. Min Act: -0.1700

Data Index: 4810884 (Open Web Text)

Max Activating Token Index: 384

Text #14

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0008. Min Act: -0.1700

Data Index: 8666243 (Open Web Text)

Max Activating Token Index: 43

Text #15

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0123. Min Act: -0.1700

Data Index: 4780986 (Open Web Text)

Max Activating Token Index: 837

Text #16

Max Range: 4.3948. Min Range: -4.3948

Max Act: 4.0070. Min Act: -0.1700

Data Index: 7464050 (Open Web Text)

Max Activating Token Index: 796

Text #17

Max Range: 4.3948. Min Range: -4.3948

Max Act: 3.9788. Min Act: -0.1700

Data Index: 5075798 (Open Web Text)

Max Activating Token Index: 624

Text #18

Max Range: 4.3948. Min Range: -4.3948

Max Act: 3.9613. Min Act: -0.1700

Data Index: 5416565 (Open Web Text)

Max Activating Token Index: 985

Text #19

Max Range: 4.3948. Min Range: -4.3948

Max Act: 3.9821. Min Act: -0.1700

Data Index: 457665 (Open Web Text)

Max Activating Token Index: 109