Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 977 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.5191. Min Act: -0.1700

Data Index: 7206378 (Open Web Text)

Max Activating Token Index: 173

Text #1

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.2754. Min Act: -0.1700

Data Index: 5816862 (Open Web Text)

Max Activating Token Index: 30

Text #2

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.1023. Min Act: -0.1700

Data Index: 6536859 (Open Web Text)

Max Activating Token Index: 881

Text #3

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.0592. Min Act: -0.1700

Data Index: 7135240 (Open Web Text)

Max Activating Token Index: 247

Text #4

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.0500. Min Act: -0.1700

Data Index: 2455046 (Open Web Text)

Max Activating Token Index: 543

Text #5

Max Range: 4.5191. Min Range: -4.5191

Max Act: 4.0100. Min Act: -0.1700

Data Index: 688481 (Open Web Text)

Max Activating Token Index: 915

Text #6

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9884. Min Act: -0.1700

Data Index: 904433 (Open Web Text)

Max Activating Token Index: 860

Text #7

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9668. Min Act: -0.1700

Data Index: 6373382 (Open Web Text)

Max Activating Token Index: 1013

Text #8

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9851. Min Act: -0.1700

Data Index: 553365 (Open Web Text)

Max Activating Token Index: 620

Text #9

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9748. Min Act: -0.1700

Data Index: 2347456 (Open Web Text)

Max Activating Token Index: 662

Text #10

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9812. Min Act: -0.1700

Data Index: 4033079 (Open Web Text)

Max Activating Token Index: 608

Text #11

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8882. Min Act: -0.1700

Data Index: 5383788 (Open Web Text)

Max Activating Token Index: 848

Text #12

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9003. Min Act: -0.1700

Data Index: 6751230 (Open Web Text)

Max Activating Token Index: 52

Text #13

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8759. Min Act: -0.1700

Data Index: 3749520 (Open Web Text)

Max Activating Token Index: 427

Text #14

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.9161. Min Act: -0.1700

Data Index: 8176594 (Open Web Text)

Max Activating Token Index: 629

Text #15

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8901. Min Act: -0.1700

Data Index: 7422979 (Open Web Text)

Max Activating Token Index: 310

Text #16

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8708. Min Act: -0.1700

Data Index: 5237318 (Open Web Text)

Max Activating Token Index: 679

Text #17

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8645. Min Act: -0.1700

Data Index: 1595389 (Open Web Text)

Max Activating Token Index: 111

Text #18

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8310. Min Act: -0.1700

Data Index: 5294793 (Open Web Text)

Max Activating Token Index: 1004

Text #19

Max Range: 4.5191. Min Range: -4.5191

Max Act: 3.8455. Min Act: -0.1700

Data Index: 6856592 (Open Web Text)

Max Activating Token Index: 325