Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2038 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.6677. Min Act: -0.1700

Data Index: 2863559 (Open Web Text)

Max Activating Token Index: 828

Text #1

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.6634. Min Act: -0.1700

Data Index: 3967747 (Open Web Text)

Max Activating Token Index: 211

Text #2

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4850. Min Act: -0.1700

Data Index: 5787889 (Open Web Text)

Max Activating Token Index: 891

Text #3

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4424. Min Act: -0.1700

Data Index: 2589771 (Open Web Text)

Max Activating Token Index: 389

Text #4

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4162. Min Act: -0.1700

Data Index: 6194962 (Open Web Text)

Max Activating Token Index: 72

Text #5

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4548. Min Act: -0.1700

Data Index: 5816060 (Open Web Text)

Max Activating Token Index: 23

Text #6

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4285. Min Act: -0.1700

Data Index: 3967746 (Open Web Text)

Max Activating Token Index: 32

Text #7

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.4286. Min Act: -0.1700

Data Index: 1634892 (Open Web Text)

Max Activating Token Index: 75

Text #8

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3934. Min Act: -0.1700

Data Index: 8527712 (Open Web Text)

Max Activating Token Index: 678

Text #9

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3664. Min Act: -0.1700

Data Index: 1917810 (Open Web Text)

Max Activating Token Index: 454

Text #10

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3561. Min Act: -0.1700

Data Index: 3934523 (Open Web Text)

Max Activating Token Index: 654

Text #11

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3332. Min Act: -0.1700

Data Index: 5285476 (Open Web Text)

Max Activating Token Index: 719

Text #12

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3387. Min Act: -0.1700

Data Index: 7467554 (Open Web Text)

Max Activating Token Index: 528

Text #13

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.2947. Min Act: -0.1700

Data Index: 2072965 (Open Web Text)

Max Activating Token Index: 307

Text #14

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3020. Min Act: -0.1700

Data Index: 2715091 (Open Web Text)

Max Activating Token Index: 691

Text #15

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3219. Min Act: -0.1700

Data Index: 292632 (Open Web Text)

Max Activating Token Index: 732

Text #16

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3103. Min Act: -0.1700

Data Index: 8513652 (Open Web Text)

Max Activating Token Index: 191

Text #17

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3078. Min Act: -0.1700

Data Index: 973593 (Open Web Text)

Max Activating Token Index: 585

Text #18

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3058. Min Act: -0.1700

Data Index: 3553483 (Open Web Text)

Max Activating Token Index: 603

Text #19

Max Range: 4.6677. Min Range: -4.6677

Max Act: 4.3135. Min Act: -0.1700

Data Index: 8757089 (Open Web Text)

Max Activating Token Index: 143