Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1522 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.5763. Min Act: -0.1700

Data Index: 7868612 (Open Web Text)

Max Activating Token Index: 421

Text #1

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.5457. Min Act: -0.1700

Data Index: 2397328 (Open Web Text)

Max Activating Token Index: 996

Text #2

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.5557. Min Act: -0.1700

Data Index: 2607842 (Open Web Text)

Max Activating Token Index: 850

Text #3

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.5262. Min Act: -0.1700

Data Index: 7041375 (Open Web Text)

Max Activating Token Index: 1009

Text #4

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4944. Min Act: -0.1700

Data Index: 7510760 (Open Web Text)

Max Activating Token Index: 790

Text #5

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4819. Min Act: -0.1700

Data Index: 4452939 (Open Web Text)

Max Activating Token Index: 563

Text #6

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4814. Min Act: -0.1700

Data Index: 4773205 (Open Web Text)

Max Activating Token Index: 719

Text #7

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4609. Min Act: -0.1700

Data Index: 8143226 (Open Web Text)

Max Activating Token Index: 637

Text #8

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4269. Min Act: -0.1700

Data Index: 2923524 (Open Web Text)

Max Activating Token Index: 97

Text #9

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4385. Min Act: -0.1700

Data Index: 6999025 (Open Web Text)

Max Activating Token Index: 950

Text #10

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4280. Min Act: -0.1700

Data Index: 6693473 (Open Web Text)

Max Activating Token Index: 885

Text #11

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4027. Min Act: -0.1700

Data Index: 7154494 (Open Web Text)

Max Activating Token Index: 432

Text #12

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4224. Min Act: -0.1700

Data Index: 5057379 (Open Web Text)

Max Activating Token Index: 472

Text #13

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4139. Min Act: -0.1700

Data Index: 4351354 (Open Web Text)

Max Activating Token Index: 268

Text #14

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.3981. Min Act: -0.1700

Data Index: 1435006 (Open Web Text)

Max Activating Token Index: 783

Text #15

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4285. Min Act: -0.1700

Data Index: 7558553 (Open Web Text)

Max Activating Token Index: 360

Text #16

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4055. Min Act: -0.1700

Data Index: 4351359 (Open Web Text)

Max Activating Token Index: 694

Text #17

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.3808. Min Act: -0.1700

Data Index: 2215873 (Open Web Text)

Max Activating Token Index: 873

Text #18

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.3922. Min Act: -0.1700

Data Index: 209282 (Open Web Text)

Max Activating Token Index: 649

Text #19

Max Range: 3.5763. Min Range: -3.5763

Max Act: 3.4020. Min Act: -0.1700

Data Index: 4660826 (Open Web Text)

Max Activating Token Index: 715