Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 828 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.7805. Min Act: -0.1700

Data Index: 408081 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.7134. Min Act: -0.1700

Data Index: 7187173 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.7090. Min Act: -0.1700

Data Index: 4294940 (Open Web Text)

Max Activating Token Index: 12

Text #3

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.7030. Min Act: -0.1700

Data Index: 4397892 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6529. Min Act: -0.1700

Data Index: 3053939 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6602. Min Act: -0.1700

Data Index: 7384574 (Open Web Text)

Max Activating Token Index: 10

Text #6

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6322. Min Act: -0.1700

Data Index: 5892586 (Open Web Text)

Max Activating Token Index: 14

Text #7

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6411. Min Act: -0.1700

Data Index: 873343 (Open Web Text)

Max Activating Token Index: 15

Text #8

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6306. Min Act: -0.1700

Data Index: 4209093 (Open Web Text)

Max Activating Token Index: 12

Text #9

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6276. Min Act: -0.1700

Data Index: 2198385 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5929. Min Act: -0.1700

Data Index: 3724561 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5973. Min Act: -0.1700

Data Index: 8440180 (Open Web Text)

Max Activating Token Index: 598

Text #12

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.6082. Min Act: -0.1700

Data Index: 5515843 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5841. Min Act: -0.1700

Data Index: 7761617 (Open Web Text)

Max Activating Token Index: 993

Text #14

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5963. Min Act: -0.1700

Data Index: 1495614 (Open Web Text)

Max Activating Token Index: 15

Text #15

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5717. Min Act: -0.1700

Data Index: 511303 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5857. Min Act: -0.1700

Data Index: 7407605 (Open Web Text)

Max Activating Token Index: 18

Text #17

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5801. Min Act: -0.1700

Data Index: 3140187 (Open Web Text)

Max Activating Token Index: 235

Text #18

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5672. Min Act: -0.1700

Data Index: 7957508 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 4.7805. Min Range: -4.7805

Max Act: 4.5562. Min Act: -0.1700

Data Index: 8108757 (Open Web Text)

Max Activating Token Index: 895