Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 24 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.5436. Min Act: -0.1700

Data Index: 748348 (Open Web Text)

Max Activating Token Index: 96

Text #1

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.4970. Min Act: -0.1700

Data Index: 8809170 (Open Web Text)

Max Activating Token Index: 694

Text #2

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.4864. Min Act: -0.1700

Data Index: 7453161 (Open Web Text)

Max Activating Token Index: 585

Text #3

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.4779. Min Act: -0.1700

Data Index: 5207738 (Open Web Text)

Max Activating Token Index: 485

Text #4

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.4420. Min Act: -0.1700

Data Index: 6844534 (Open Web Text)

Max Activating Token Index: 527

Text #5

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3878. Min Act: -0.1700

Data Index: 339637 (Open Web Text)

Max Activating Token Index: 73

Text #6

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3523. Min Act: -0.1700

Data Index: 6478188 (Open Web Text)

Max Activating Token Index: 309

Text #7

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3329. Min Act: -0.1700

Data Index: 6879000 (Open Web Text)

Max Activating Token Index: 182

Text #8

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3403. Min Act: -0.1700

Data Index: 7599122 (Open Web Text)

Max Activating Token Index: 621

Text #9

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3230. Min Act: -0.1700

Data Index: 4836053 (Open Web Text)

Max Activating Token Index: 1008

Text #10

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3020. Min Act: -0.1700

Data Index: 5203679 (Open Web Text)

Max Activating Token Index: 176

Text #11

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3053. Min Act: -0.1700

Data Index: 4374002 (Open Web Text)

Max Activating Token Index: 53

Text #12

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3058. Min Act: -0.1700

Data Index: 5387315 (Open Web Text)

Max Activating Token Index: 240

Text #13

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2922. Min Act: -0.1700

Data Index: 7488542 (Open Web Text)

Max Activating Token Index: 162

Text #14

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.3095. Min Act: -0.1700

Data Index: 7025332 (Open Web Text)

Max Activating Token Index: 77

Text #15

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2758. Min Act: -0.1700

Data Index: 6661479 (Open Web Text)

Max Activating Token Index: 591

Text #16

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2734. Min Act: -0.1700

Data Index: 1305571 (Open Web Text)

Max Activating Token Index: 995

Text #17

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2530. Min Act: -0.1700

Data Index: 26556 (Open Web Text)

Max Activating Token Index: 949

Text #18

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2866. Min Act: -0.1700

Data Index: 4288259 (Open Web Text)

Max Activating Token Index: 899

Text #19

Max Range: 3.5436. Min Range: -3.5436

Max Act: 3.2657. Min Act: -0.1700

Data Index: 2628693 (Open Web Text)

Max Activating Token Index: 536