Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 463 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.4867. Min Act: -0.1573

Data Index: 4117671 (Open Web Text)

Max Activating Token Index: 578

Text #1

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3890. Min Act: -0.0698

Data Index: 4002878 (Open Web Text)

Max Activating Token Index: 558

Text #2

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3682. Min Act: -0.0294

Data Index: 7478233 (Open Web Text)

Max Activating Token Index: 721

Text #3

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3300. Min Act: -0.1173

Data Index: 1347222 (Open Web Text)

Max Activating Token Index: 547

Text #4

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3121. Min Act: -0.1499

Data Index: 8310784 (Open Web Text)

Max Activating Token Index: 634

Text #5

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3212. Min Act: -0.1014

Data Index: 5744234 (Open Web Text)

Max Activating Token Index: 543

Text #6

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2998. Min Act: -0.1301

Data Index: 6438728 (Open Web Text)

Max Activating Token Index: 591

Text #7

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2976. Min Act: -0.1671

Data Index: 3185498 (Open Web Text)

Max Activating Token Index: 590

Text #8

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2899. Min Act: -0.1085

Data Index: 661995 (Open Web Text)

Max Activating Token Index: 560

Text #9

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2876. Min Act: -0.1671

Data Index: 6386007 (Open Web Text)

Max Activating Token Index: 571

Text #10

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.3061. Min Act: -0.1151

Data Index: 250325 (Open Web Text)

Max Activating Token Index: 584

Text #11

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2911. Min Act: -0.0836

Data Index: 282684 (Open Web Text)

Max Activating Token Index: 471

Text #12

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2937. Min Act: -0.0934

Data Index: 3955459 (Open Web Text)

Max Activating Token Index: 587

Text #13

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2767. Min Act: -0.0971

Data Index: 3754891 (Open Web Text)

Max Activating Token Index: 497

Text #14

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2785. Min Act: -0.1549

Data Index: 3424658 (Open Web Text)

Max Activating Token Index: 534

Text #15

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2959. Min Act: -0.1566

Data Index: 8748363 (Open Web Text)

Max Activating Token Index: 624

Text #16

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2819. Min Act: -0.0738

Data Index: 5425264 (Open Web Text)

Max Activating Token Index: 559

Text #17

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2746. Min Act: -0.1526

Data Index: 5084137 (Open Web Text)

Max Activating Token Index: 572

Text #18

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2753. Min Act: -0.1321

Data Index: 230213 (Open Web Text)

Max Activating Token Index: 606

Text #19

Max Range: 2.4867. Min Range: -2.4867

Max Act: 2.2941. Min Act: -0.0425

Data Index: 7478234 (Open Web Text)

Max Activating Token Index: 808