Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1169 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1108. Min Range: -5.1108

Max Act: 5.1108. Min Act: -0.1700

Data Index: 7886971 (Open Web Text)

Max Activating Token Index: 798

Text #1

Max Range: 5.1108. Min Range: -5.1108

Max Act: 5.0407. Min Act: -0.1700

Data Index: 3498115 (Open Web Text)

Max Activating Token Index: 272

Text #2

Max Range: 5.1108. Min Range: -5.1108

Max Act: 5.0094. Min Act: -0.1700

Data Index: 1130817 (Open Web Text)

Max Activating Token Index: 890

Text #3

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9920. Min Act: -0.1700

Data Index: 4742402 (Open Web Text)

Max Activating Token Index: 823

Text #4

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9718. Min Act: -0.1700

Data Index: 5123325 (Open Web Text)

Max Activating Token Index: 473

Text #5

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9434. Min Act: -0.1700

Data Index: 8116620 (Open Web Text)

Max Activating Token Index: 338

Text #6

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8997. Min Act: -0.1700

Data Index: 4042954 (Open Web Text)

Max Activating Token Index: 334

Text #7

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8859. Min Act: -0.1700

Data Index: 5816607 (Open Web Text)

Max Activating Token Index: 797

Text #8

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8998. Min Act: -0.1700

Data Index: 3451553 (Open Web Text)

Max Activating Token Index: 999

Text #9

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9110. Min Act: -0.1700

Data Index: 5244467 (Open Web Text)

Max Activating Token Index: 154

Text #10

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8811. Min Act: -0.1700

Data Index: 5132866 (Open Web Text)

Max Activating Token Index: 292

Text #11

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9212. Min Act: -0.1700

Data Index: 4519227 (Open Web Text)

Max Activating Token Index: 806

Text #12

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9024. Min Act: -0.1700

Data Index: 1105144 (Open Web Text)

Max Activating Token Index: 703

Text #13

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.9078. Min Act: -0.1700

Data Index: 258055 (Open Web Text)

Max Activating Token Index: 597

Text #14

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8667. Min Act: -0.1700

Data Index: 5725125 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8509. Min Act: -0.1700

Data Index: 3717823 (Open Web Text)

Max Activating Token Index: 872

Text #16

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8384. Min Act: -0.1700

Data Index: 503575 (Open Web Text)

Max Activating Token Index: 686

Text #17

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8537. Min Act: -0.1700

Data Index: 4548775 (Open Web Text)

Max Activating Token Index: 564

Text #18

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8542. Min Act: -0.1700

Data Index: 991111 (Open Web Text)

Max Activating Token Index: 108

Text #19

Max Range: 5.1108. Min Range: -5.1108

Max Act: 4.8881. Min Act: -0.1700

Data Index: 7170309 (Open Web Text)

Max Activating Token Index: 358