Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 457 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.5164. Min Act: -0.1700

Data Index: 801567 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.5259. Min Act: -0.1700

Data Index: 3768355 (Open Web Text)

Max Activating Token Index: 642

Text #2

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.5360. Min Act: -0.1700

Data Index: 6555192 (Open Web Text)

Max Activating Token Index: 938

Text #3

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4595. Min Act: -0.1696

Data Index: 1484731 (Open Web Text)

Max Activating Token Index: 355

Text #4

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4683. Min Act: -0.1700

Data Index: 1011441 (Open Web Text)

Max Activating Token Index: 321

Text #5

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4751. Min Act: -0.1700

Data Index: 3921674 (Open Web Text)

Max Activating Token Index: 140

Text #6

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4655. Min Act: -0.1700

Data Index: 1586027 (Open Web Text)

Max Activating Token Index: 699

Text #7

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4402. Min Act: -0.1700

Data Index: 7702710 (Open Web Text)

Max Activating Token Index: 449

Text #8

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4197. Min Act: -0.1700

Data Index: 6433027 (Open Web Text)

Max Activating Token Index: 12

Text #9

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4050. Min Act: -0.1700

Data Index: 3111170 (Open Web Text)

Max Activating Token Index: 952

Text #10

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3946. Min Act: -0.1700

Data Index: 716321 (Open Web Text)

Max Activating Token Index: 135

Text #11

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4004. Min Act: -0.1700

Data Index: 663006 (Open Web Text)

Max Activating Token Index: 1000

Text #12

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.4009. Min Act: -0.1700

Data Index: 5043734 (Open Web Text)

Max Activating Token Index: 260

Text #13

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3661. Min Act: -0.1700

Data Index: 3779315 (Open Web Text)

Max Activating Token Index: 487

Text #14

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3603. Min Act: -0.1700

Data Index: 5377631 (Open Web Text)

Max Activating Token Index: 285

Text #15

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3580. Min Act: -0.1700

Data Index: 4320891 (Open Web Text)

Max Activating Token Index: 791

Text #16

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3662. Min Act: -0.1700

Data Index: 2823214 (Open Web Text)

Max Activating Token Index: 68

Text #17

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3749. Min Act: -0.1700

Data Index: 7685474 (Open Web Text)

Max Activating Token Index: 58

Text #18

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3563. Min Act: -0.1700

Data Index: 2851235 (Open Web Text)

Max Activating Token Index: 73

Text #19

Max Range: 5.5360. Min Range: -5.5360

Max Act: 5.3690. Min Act: -0.1700

Data Index: 7685475 (Open Web Text)

Max Activating Token Index: 125