Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 360 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.2808. Min Act: -0.1700

Data Index: 7121445 (Open Web Text)

Max Activating Token Index: 158

Text #1

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.2325. Min Act: -0.1700

Data Index: 2207395 (Open Web Text)

Max Activating Token Index: 366

Text #2

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.1462. Min Act: -0.1700

Data Index: 8697206 (Open Web Text)

Max Activating Token Index: 580

Text #3

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.1128. Min Act: -0.1700

Data Index: 5689715 (Open Web Text)

Max Activating Token Index: 701

Text #4

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0905. Min Act: -0.1700

Data Index: 5512102 (Open Web Text)

Max Activating Token Index: 209

Text #5

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0816. Min Act: -0.1700

Data Index: 1481130 (Open Web Text)

Max Activating Token Index: 800

Text #6

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0974. Min Act: -0.1700

Data Index: 3151231 (Open Web Text)

Max Activating Token Index: 643

Text #7

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.1071. Min Act: -0.1700

Data Index: 7641599 (Open Web Text)

Max Activating Token Index: 933

Text #8

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0705. Min Act: -0.1700

Data Index: 8025157 (Open Web Text)

Max Activating Token Index: 285

Text #9

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0707. Min Act: -0.1700

Data Index: 5239697 (Open Web Text)

Max Activating Token Index: 404

Text #10

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0747. Min Act: -0.1700

Data Index: 7690593 (Open Web Text)

Max Activating Token Index: 490

Text #11

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0573. Min Act: -0.1700

Data Index: 7602322 (Open Web Text)

Max Activating Token Index: 657

Text #12

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0698. Min Act: -0.1700

Data Index: 1978555 (Open Web Text)

Max Activating Token Index: 894

Text #13

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0649. Min Act: -0.1700

Data Index: 1646951 (Open Web Text)

Max Activating Token Index: 495

Text #14

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0595. Min Act: -0.1700

Data Index: 4671063 (Open Web Text)

Max Activating Token Index: 654

Text #15

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0393. Min Act: -0.1700

Data Index: 4996611 (Open Web Text)

Max Activating Token Index: 795

Text #16

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0307. Min Act: -0.1700

Data Index: 5120688 (Open Web Text)

Max Activating Token Index: 651

Text #17

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0315. Min Act: -0.1700

Data Index: 7285056 (Open Web Text)

Max Activating Token Index: 895

Text #18

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0402. Min Act: -0.1700

Data Index: 896154 (Open Web Text)

Max Activating Token Index: 158

Text #19

Max Range: 4.2808. Min Range: -4.2808

Max Act: 4.0583. Min Act: -0.1699

Data Index: 4212182 (Open Web Text)

Max Activating Token Index: 951