Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2112 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.4817. Min Act: -0.1700

Data Index: 1655669 (Open Web Text)

Max Activating Token Index: 446

Text #1

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.4310. Min Act: -0.1700

Data Index: 8008303 (Open Web Text)

Max Activating Token Index: 870

Text #2

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.4083. Min Act: -0.1700

Data Index: 1275986 (Open Web Text)

Max Activating Token Index: 925

Text #3

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.4026. Min Act: -0.1700

Data Index: 2837478 (Open Web Text)

Max Activating Token Index: 635

Text #4

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3956. Min Act: -0.1700

Data Index: 4394659 (Open Web Text)

Max Activating Token Index: 168

Text #5

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3966. Min Act: -0.1700

Data Index: 599132 (Open Web Text)

Max Activating Token Index: 784

Text #6

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.4105. Min Act: -0.1700

Data Index: 139503 (Open Web Text)

Max Activating Token Index: 94

Text #7

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3928. Min Act: -0.1700

Data Index: 6489017 (Open Web Text)

Max Activating Token Index: 727

Text #8

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3869. Min Act: -0.1700

Data Index: 8241286 (Open Web Text)

Max Activating Token Index: 241

Text #9

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3501. Min Act: -0.1700

Data Index: 8301168 (Open Web Text)

Max Activating Token Index: 993

Text #10

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3619. Min Act: -0.1700

Data Index: 6607190 (Open Web Text)

Max Activating Token Index: 529

Text #11

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3606. Min Act: -0.1700

Data Index: 7853176 (Open Web Text)

Max Activating Token Index: 293

Text #12

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3435. Min Act: -0.1700

Data Index: 3104099 (Open Web Text)

Max Activating Token Index: 478

Text #13

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3480. Min Act: -0.1700

Data Index: 5381995 (Open Web Text)

Max Activating Token Index: 628

Text #14

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3545. Min Act: -0.1700

Data Index: 8103286 (Open Web Text)

Max Activating Token Index: 467

Text #15

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3320. Min Act: -0.1700

Data Index: 7226575 (Open Web Text)

Max Activating Token Index: 565

Text #16

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3253. Min Act: -0.1700

Data Index: 8387741 (Open Web Text)

Max Activating Token Index: 294

Text #17

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3394. Min Act: -0.1700

Data Index: 6319594 (Open Web Text)

Max Activating Token Index: 977

Text #18

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3265. Min Act: -0.1700

Data Index: 4918613 (Open Web Text)

Max Activating Token Index: 597

Text #19

Max Range: 3.4817. Min Range: -3.4817

Max Act: 3.3195. Min Act: -0.1700

Data Index: 1050203 (Open Web Text)

Max Activating Token Index: 317