Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 24 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8619. Min Act: -0.1700

Data Index: 6392001 (Open Web Text)

Max Activating Token Index: 417

Text #1

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8372. Min Act: -0.1700

Data Index: 6236988 (Open Web Text)

Max Activating Token Index: 610

Text #2

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8322. Min Act: -0.1700

Data Index: 5508450 (Open Web Text)

Max Activating Token Index: 805

Text #3

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8128. Min Act: -0.1700

Data Index: 3881098 (Open Web Text)

Max Activating Token Index: 403

Text #4

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8265. Min Act: -0.1700

Data Index: 3791004 (Open Web Text)

Max Activating Token Index: 17

Text #5

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8031. Min Act: -0.1700

Data Index: 5783417 (Open Web Text)

Max Activating Token Index: 436

Text #6

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7785. Min Act: -0.1700

Data Index: 2482693 (Open Web Text)

Max Activating Token Index: 924

Text #7

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7899. Min Act: -0.1700

Data Index: 3287924 (Open Web Text)

Max Activating Token Index: 853

Text #8

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7895. Min Act: -0.1700

Data Index: 1612755 (Open Web Text)

Max Activating Token Index: 414

Text #9

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.8020. Min Act: -0.1700

Data Index: 2808014 (Open Web Text)

Max Activating Token Index: 980

Text #10

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7779. Min Act: -0.1700

Data Index: 1414360 (Open Web Text)

Max Activating Token Index: 994

Text #11

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7392. Min Act: -0.1700

Data Index: 3474564 (Open Web Text)

Max Activating Token Index: 984

Text #12

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7837. Min Act: -0.1700

Data Index: 3640088 (Open Web Text)

Max Activating Token Index: 312

Text #13

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7787. Min Act: -0.1700

Data Index: 4621157 (Open Web Text)

Max Activating Token Index: 775

Text #14

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7652. Min Act: -0.1700

Data Index: 4728933 (Open Web Text)

Max Activating Token Index: 430

Text #15

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7445. Min Act: -0.1700

Data Index: 377919 (Open Web Text)

Max Activating Token Index: 902

Text #16

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7762. Min Act: -0.1700

Data Index: 5305644 (Open Web Text)

Max Activating Token Index: 138

Text #17

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7622. Min Act: -0.1700

Data Index: 1176844 (Open Web Text)

Max Activating Token Index: 7

Text #18

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7447. Min Act: -0.1700

Data Index: 5320967 (Open Web Text)

Max Activating Token Index: 770

Text #19

Max Range: 3.8619. Min Range: -3.8619

Max Act: 3.7629. Min Act: -0.1700

Data Index: 5714555 (Open Web Text)

Max Activating Token Index: 778