Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2193 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.5795. Min Act: -0.1700

Data Index: 8469223 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.5290. Min Act: -0.1700

Data Index: 8135141 (Open Web Text)

Max Activating Token Index: 145

Text #2

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4864. Min Act: -0.1700

Data Index: 7512713 (Open Web Text)

Max Activating Token Index: 266

Text #3

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4760. Min Act: -0.1700

Data Index: 2617952 (Open Web Text)

Max Activating Token Index: 843

Text #4

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4437. Min Act: -0.1700

Data Index: 6176256 (Open Web Text)

Max Activating Token Index: 62

Text #5

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4627. Min Act: -0.1700

Data Index: 1107647 (Open Web Text)

Max Activating Token Index: 169

Text #6

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4480. Min Act: -0.1700

Data Index: 7736796 (Open Web Text)

Max Activating Token Index: 962

Text #7

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4489. Min Act: -0.1700

Data Index: 761516 (Open Web Text)

Max Activating Token Index: 652

Text #8

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4716. Min Act: -0.1700

Data Index: 5878588 (Open Web Text)

Max Activating Token Index: 364

Text #9

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4652. Min Act: -0.1700

Data Index: 2719490 (Open Web Text)

Max Activating Token Index: 128

Text #10

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4681. Min Act: -0.1700

Data Index: 5594827 (Open Web Text)

Max Activating Token Index: 191

Text #11

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4630. Min Act: -0.1700

Data Index: 4319697 (Open Web Text)

Max Activating Token Index: 285

Text #12

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4550. Min Act: -0.1700

Data Index: 4016080 (Open Web Text)

Max Activating Token Index: 659

Text #13

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4190. Min Act: -0.1700

Data Index: 6747818 (Open Web Text)

Max Activating Token Index: 581

Text #14

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4089. Min Act: -0.1700

Data Index: 7101794 (Open Web Text)

Max Activating Token Index: 155

Text #15

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4360. Min Act: -0.1700

Data Index: 4727944 (Open Web Text)

Max Activating Token Index: 131

Text #16

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4224. Min Act: -0.1700

Data Index: 7543891 (Open Web Text)

Max Activating Token Index: 142

Text #17

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4446. Min Act: -0.1700

Data Index: 1945597 (Open Web Text)

Max Activating Token Index: 170

Text #18

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.4393. Min Act: -0.1700

Data Index: 1747420 (Open Web Text)

Max Activating Token Index: 250

Text #19

Max Range: 3.5795. Min Range: -3.5795

Max Act: 3.3804. Min Act: -0.1700

Data Index: 382976 (Open Web Text)

Max Activating Token Index: 647