Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1917 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0145. Min Range: -4.0145

Max Act: 4.0145. Min Act: -0.1700

Data Index: 6799179 (Open Web Text)

Max Activating Token Index: 527

Text #1

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9661. Min Act: -0.1700

Data Index: 274389 (Open Web Text)

Max Activating Token Index: 491

Text #2

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9373. Min Act: -0.1700

Data Index: 5282563 (Open Web Text)

Max Activating Token Index: 98

Text #3

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9364. Min Act: -0.1700

Data Index: 5389206 (Open Web Text)

Max Activating Token Index: 562

Text #4

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9601. Min Act: -0.1700

Data Index: 8448388 (Open Web Text)

Max Activating Token Index: 175

Text #5

Max Range: 4.0145. Min Range: -4.0145

Max Act: 4.0112. Min Act: -0.1700

Data Index: 5597959 (Open Web Text)

Max Activating Token Index: 697

Text #6

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9049. Min Act: -0.1700

Data Index: 7275566 (Open Web Text)

Max Activating Token Index: 584

Text #7

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9704. Min Act: -0.1700

Data Index: 1166638 (Open Web Text)

Max Activating Token Index: 502

Text #8

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9895. Min Act: -0.1700

Data Index: 4580723 (Open Web Text)

Max Activating Token Index: 844

Text #9

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9706. Min Act: -0.1700

Data Index: 6845420 (Open Web Text)

Max Activating Token Index: 477

Text #10

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.8514. Min Act: -0.1700

Data Index: 427086 (Open Web Text)

Max Activating Token Index: 923

Text #11

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9104. Min Act: -0.1700

Data Index: 2258708 (Open Web Text)

Max Activating Token Index: 775

Text #12

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9286. Min Act: -0.1700

Data Index: 152255 (Open Web Text)

Max Activating Token Index: 530

Text #13

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9308. Min Act: -0.1700

Data Index: 8007207 (Open Web Text)

Max Activating Token Index: 699

Text #14

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9069. Min Act: -0.1700

Data Index: 7692019 (Open Web Text)

Max Activating Token Index: 468

Text #15

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9182. Min Act: -0.1700

Data Index: 5791895 (Open Web Text)

Max Activating Token Index: 92

Text #16

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9186. Min Act: -0.1700

Data Index: 5712362 (Open Web Text)

Max Activating Token Index: 844

Text #17

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9081. Min Act: -0.1700

Data Index: 5028702 (Open Web Text)

Max Activating Token Index: 340

Text #18

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.9134. Min Act: -0.1700

Data Index: 877896 (Open Web Text)

Max Activating Token Index: 431

Text #19

Max Range: 4.0145. Min Range: -4.0145

Max Act: 3.8267. Min Act: -0.1700

Data Index: 2082105 (Open Web Text)

Max Activating Token Index: 426