Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 621 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0265. Min Range: -3.0265

Max Act: 3.0265. Min Act: -0.1700

Data Index: 4936148 (Open Web Text)

Max Activating Token Index: 684

Text #1

Max Range: 3.0265. Min Range: -3.0265

Max Act: 3.0196. Min Act: -0.1700

Data Index: 5456005 (Open Web Text)

Max Activating Token Index: 192

Text #2

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9971. Min Act: -0.1700

Data Index: 2995985 (Open Web Text)

Max Activating Token Index: 744

Text #3

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9570. Min Act: -0.1700

Data Index: 7519479 (Open Web Text)

Max Activating Token Index: 709

Text #4

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9531. Min Act: -0.1700

Data Index: 3458347 (Open Web Text)

Max Activating Token Index: 879

Text #5

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9572. Min Act: -0.1700

Data Index: 8027733 (Open Web Text)

Max Activating Token Index: 585

Text #6

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9195. Min Act: -0.1700

Data Index: 4156102 (Open Web Text)

Max Activating Token Index: 925

Text #7

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9083. Min Act: -0.1700

Data Index: 5123180 (Open Web Text)

Max Activating Token Index: 307

Text #8

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9011. Min Act: -0.1700

Data Index: 3286648 (Open Web Text)

Max Activating Token Index: 671

Text #9

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8661. Min Act: -0.1700

Data Index: 7247958 (Open Web Text)

Max Activating Token Index: 265

Text #10

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9023. Min Act: -0.1700

Data Index: 5029898 (Open Web Text)

Max Activating Token Index: 648

Text #11

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.9198. Min Act: -0.1700

Data Index: 6084850 (Open Web Text)

Max Activating Token Index: 545

Text #12

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8131. Min Act: -0.1700

Data Index: 2971666 (Open Web Text)

Max Activating Token Index: 847

Text #13

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8623. Min Act: -0.1700

Data Index: 5456004 (Open Web Text)

Max Activating Token Index: 888

Text #14

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8594. Min Act: -0.1700

Data Index: 7180063 (Open Web Text)

Max Activating Token Index: 1005

Text #15

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8425. Min Act: -0.1700

Data Index: 8497417 (Open Web Text)

Max Activating Token Index: 720

Text #16

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8111. Min Act: -0.1700

Data Index: 3538974 (Open Web Text)

Max Activating Token Index: 381

Text #17

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8002. Min Act: -0.1700

Data Index: 7970168 (Open Web Text)

Max Activating Token Index: 370

Text #18

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8717. Min Act: -0.1700

Data Index: 2536454 (Open Web Text)

Max Activating Token Index: 573

Text #19

Max Range: 3.0265. Min Range: -3.0265

Max Act: 2.8305. Min Act: -0.1700

Data Index: 6866258 (Open Web Text)

Max Activating Token Index: 446