Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 421 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.5780. Min Act: -0.1700

Data Index: 2618340 (Open Web Text)

Max Activating Token Index: 688

Text #1

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.5248. Min Act: -0.1700

Data Index: 3165556 (Open Web Text)

Max Activating Token Index: 374

Text #2

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4466. Min Act: -0.1700

Data Index: 6119078 (Open Web Text)

Max Activating Token Index: 254

Text #3

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4612. Min Act: -0.1700

Data Index: 3383379 (Open Web Text)

Max Activating Token Index: 605

Text #4

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4375. Min Act: -0.1700

Data Index: 6196358 (Open Web Text)

Max Activating Token Index: 425

Text #5

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4249. Min Act: -0.1700

Data Index: 7464671 (Open Web Text)

Max Activating Token Index: 582

Text #6

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3956. Min Act: -0.1700

Data Index: 7427835 (Open Web Text)

Max Activating Token Index: 669

Text #7

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4115. Min Act: -0.1700

Data Index: 6205243 (Open Web Text)

Max Activating Token Index: 989

Text #8

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3937. Min Act: -0.1700

Data Index: 5756279 (Open Web Text)

Max Activating Token Index: 912

Text #9

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3984. Min Act: -0.1700

Data Index: 7258355 (Open Web Text)

Max Activating Token Index: 907

Text #10

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3934. Min Act: -0.1700

Data Index: 5098657 (Open Web Text)

Max Activating Token Index: 804

Text #11

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3881. Min Act: -0.1700

Data Index: 3736970 (Open Web Text)

Max Activating Token Index: 923

Text #12

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3876. Min Act: -0.1700

Data Index: 1581767 (Open Web Text)

Max Activating Token Index: 131

Text #13

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.4078. Min Act: -0.1700

Data Index: 6258439 (Open Web Text)

Max Activating Token Index: 378

Text #14

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3861. Min Act: -0.1700

Data Index: 2740724 (Open Web Text)

Max Activating Token Index: 825

Text #15

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3870. Min Act: -0.1700

Data Index: 8580151 (Open Web Text)

Max Activating Token Index: 589

Text #16

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3979. Min Act: -0.1700

Data Index: 6115511 (Open Web Text)

Max Activating Token Index: 421

Text #17

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3556. Min Act: -0.1700

Data Index: 2078275 (Open Web Text)

Max Activating Token Index: 422

Text #18

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3773. Min Act: -0.1700

Data Index: 1427274 (Open Web Text)

Max Activating Token Index: 454

Text #19

Max Range: 3.5780. Min Range: -3.5780

Max Act: 3.3783. Min Act: -0.1700

Data Index: 178364 (Open Web Text)

Max Activating Token Index: 386