Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1639 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.5891. Min Act: -0.1700

Data Index: 568136 (Open Web Text)

Max Activating Token Index: 380

Text #1

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.5415. Min Act: -0.1700

Data Index: 8175594 (Open Web Text)

Max Activating Token Index: 97

Text #2

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.4815. Min Act: -0.1700

Data Index: 8242000 (Open Web Text)

Max Activating Token Index: 582

Text #3

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.4263. Min Act: -0.1700

Data Index: 673324 (Open Web Text)

Max Activating Token Index: 440

Text #4

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3965. Min Act: -0.1700

Data Index: 3969282 (Open Web Text)

Max Activating Token Index: 448

Text #5

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3627. Min Act: -0.1700

Data Index: 4340846 (Open Web Text)

Max Activating Token Index: 537

Text #6

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3253. Min Act: -0.1700

Data Index: 2175165 (Open Web Text)

Max Activating Token Index: 512

Text #7

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3158. Min Act: -0.1700

Data Index: 2654070 (Open Web Text)

Max Activating Token Index: 120

Text #8

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3347. Min Act: -0.1700

Data Index: 3666438 (Open Web Text)

Max Activating Token Index: 236

Text #9

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.3514. Min Act: -0.1700

Data Index: 775257 (Open Web Text)

Max Activating Token Index: 533

Text #10

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2536. Min Act: -0.1700

Data Index: 6953679 (Open Web Text)

Max Activating Token Index: 70

Text #11

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2308. Min Act: -0.1700

Data Index: 1982715 (Open Web Text)

Max Activating Token Index: 618

Text #12

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2489. Min Act: -0.1700

Data Index: 5352168 (Open Web Text)

Max Activating Token Index: 584

Text #13

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2542. Min Act: -0.1700

Data Index: 5545152 (Open Web Text)

Max Activating Token Index: 949

Text #14

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.1875. Min Act: -0.1700

Data Index: 1778842 (Open Web Text)

Max Activating Token Index: 437

Text #15

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.1731. Min Act: -0.1700

Data Index: 5058277 (Open Web Text)

Max Activating Token Index: 98

Text #16

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2155. Min Act: -0.1700

Data Index: 4881164 (Open Web Text)

Max Activating Token Index: 500

Text #17

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2296. Min Act: -0.1700

Data Index: 5857501 (Open Web Text)

Max Activating Token Index: 143

Text #18

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.2111. Min Act: -0.1700

Data Index: 4578949 (Open Web Text)

Max Activating Token Index: 605

Text #19

Max Range: 4.5891. Min Range: -4.5891

Max Act: 4.1917. Min Act: -0.1700

Data Index: 4924756 (Open Web Text)

Max Activating Token Index: 915