Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2528 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3872. Min Act: -0.1700

Data Index: 6558533 (Open Web Text)

Max Activating Token Index: 579

Text #1

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3945. Min Act: -0.1700

Data Index: 6558535 (Open Web Text)

Max Activating Token Index: 619

Text #2

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3785. Min Act: -0.1700

Data Index: 396429 (Open Web Text)

Max Activating Token Index: 599

Text #3

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3433. Min Act: -0.1700

Data Index: 7630502 (Open Web Text)

Max Activating Token Index: 247

Text #4

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3663. Min Act: -0.1700

Data Index: 2815221 (Open Web Text)

Max Activating Token Index: 819

Text #5

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3601. Min Act: -0.1700

Data Index: 7630500 (Open Web Text)

Max Activating Token Index: 207

Text #6

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3580. Min Act: -0.1700

Data Index: 3164702 (Open Web Text)

Max Activating Token Index: 453

Text #7

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3794. Min Act: -0.1700

Data Index: 396427 (Open Web Text)

Max Activating Token Index: 559

Text #8

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3566. Min Act: -0.1700

Data Index: 2470412 (Open Web Text)

Max Activating Token Index: 849

Text #9

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3710. Min Act: -0.1700

Data Index: 3164704 (Open Web Text)

Max Activating Token Index: 493

Text #10

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.3380. Min Act: -0.1700

Data Index: 2976899 (Open Web Text)

Max Activating Token Index: 249

Text #11

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2853. Min Act: -0.1700

Data Index: 1557408 (Open Web Text)

Max Activating Token Index: 699

Text #12

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2778. Min Act: -0.1700

Data Index: 5549617 (Open Web Text)

Max Activating Token Index: 605

Text #13

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2608. Min Act: -0.1700

Data Index: 5883534 (Open Web Text)

Max Activating Token Index: 96

Text #14

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2818. Min Act: -0.1700

Data Index: 5567838 (Open Web Text)

Max Activating Token Index: 621

Text #15

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2814. Min Act: -0.1700

Data Index: 7876897 (Open Web Text)

Max Activating Token Index: 173

Text #16

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2410. Min Act: -0.1700

Data Index: 4216162 (Open Web Text)

Max Activating Token Index: 576

Text #17

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2327. Min Act: -0.1700

Data Index: 5731566 (Open Web Text)

Max Activating Token Index: 535

Text #18

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2114. Min Act: -0.1700

Data Index: 1032722 (Open Web Text)

Max Activating Token Index: 206

Text #19

Max Range: 5.3945. Min Range: -5.3945

Max Act: 5.2405. Min Act: -0.1700

Data Index: 3662797 (Open Web Text)

Max Activating Token Index: 738