Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2194 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.4290. Min Act: -0.1692

Data Index: 3606102 (Open Web Text)

Max Activating Token Index: 671

Text #1

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.4318. Min Act: -0.1700

Data Index: 7062001 (Open Web Text)

Max Activating Token Index: 804

Text #2

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.4085. Min Act: -0.1696

Data Index: 1602416 (Open Web Text)

Max Activating Token Index: 796

Text #3

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3966. Min Act: -0.1700

Data Index: 6588393 (Open Web Text)

Max Activating Token Index: 378

Text #4

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3907. Min Act: -0.1693

Data Index: 2450081 (Open Web Text)

Max Activating Token Index: 761

Text #5

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3875. Min Act: -0.1674

Data Index: 4013540 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3907. Min Act: -0.1622

Data Index: 3909284 (Open Web Text)

Max Activating Token Index: 721

Text #7

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3667. Min Act: -0.1695

Data Index: 3041490 (Open Web Text)

Max Activating Token Index: 617

Text #8

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3736. Min Act: -0.1699

Data Index: 4590131 (Open Web Text)

Max Activating Token Index: 762

Text #9

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3818. Min Act: -0.1698

Data Index: 2015186 (Open Web Text)

Max Activating Token Index: 903

Text #10

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3510. Min Act: -0.1674

Data Index: 766815 (Open Web Text)

Max Activating Token Index: 644

Text #11

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3563. Min Act: -0.1654

Data Index: 7642154 (Open Web Text)

Max Activating Token Index: 786

Text #12

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3598. Min Act: -0.1697

Data Index: 6179720 (Open Web Text)

Max Activating Token Index: 805

Text #13

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3544. Min Act: -0.1648

Data Index: 7077716 (Open Web Text)

Max Activating Token Index: 785

Text #14

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3502. Min Act: -0.1696

Data Index: 3429911 (Open Web Text)

Max Activating Token Index: 107

Text #15

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3434. Min Act: -0.1684

Data Index: 3633125 (Open Web Text)

Max Activating Token Index: 513

Text #16

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3449. Min Act: -0.1698

Data Index: 3890957 (Open Web Text)

Max Activating Token Index: 602

Text #17

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3431. Min Act: -0.1688

Data Index: 5287189 (Open Web Text)

Max Activating Token Index: 158

Text #18

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3479. Min Act: -0.1699

Data Index: 2420117 (Open Web Text)

Max Activating Token Index: 24

Text #19

Max Range: 0.4318. Min Range: -0.4318

Max Act: 0.3338. Min Act: -0.1696

Data Index: 35150 (Open Web Text)

Max Activating Token Index: 843