Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2433 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8949. Min Act: -0.1700

Data Index: 1046052 (Open Web Text)

Max Activating Token Index: 696

Text #1

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.9066. Min Act: -0.1700

Data Index: 3436521 (Open Web Text)

Max Activating Token Index: 871

Text #2

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8812. Min Act: -0.1700

Data Index: 7970915 (Open Web Text)

Max Activating Token Index: 89

Text #3

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8312. Min Act: -0.1700

Data Index: 1313138 (Open Web Text)

Max Activating Token Index: 837

Text #4

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8429. Min Act: -0.1700

Data Index: 2934469 (Open Web Text)

Max Activating Token Index: 773

Text #5

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8393. Min Act: -0.1700

Data Index: 7940530 (Open Web Text)

Max Activating Token Index: 823

Text #6

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8301. Min Act: -0.1700

Data Index: 6308457 (Open Web Text)

Max Activating Token Index: 636

Text #7

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8660. Min Act: -0.1700

Data Index: 623436 (Open Web Text)

Max Activating Token Index: 143

Text #8

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8569. Min Act: -0.1700

Data Index: 3600745 (Open Web Text)

Max Activating Token Index: 730

Text #9

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8834. Min Act: -0.1700

Data Index: 1818611 (Open Web Text)

Max Activating Token Index: 175

Text #10

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8138. Min Act: -0.1700

Data Index: 2435194 (Open Web Text)

Max Activating Token Index: 549

Text #11

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8091. Min Act: -0.1700

Data Index: 1476469 (Open Web Text)

Max Activating Token Index: 20

Text #12

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.7948. Min Act: -0.1700

Data Index: 2625304 (Open Web Text)

Max Activating Token Index: 710

Text #13

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8200. Min Act: -0.1700

Data Index: 2074052 (Open Web Text)

Max Activating Token Index: 287

Text #14

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8197. Min Act: -0.1700

Data Index: 7531280 (Open Web Text)

Max Activating Token Index: 201

Text #15

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.7669. Min Act: -0.1700

Data Index: 68245 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.7761. Min Act: -0.1700

Data Index: 2263209 (Open Web Text)

Max Activating Token Index: 851

Text #17

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.8054. Min Act: -0.1700

Data Index: 4033478 (Open Web Text)

Max Activating Token Index: 325

Text #18

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.7670. Min Act: -0.1700

Data Index: 50238 (Open Web Text)

Max Activating Token Index: 664

Text #19

Max Range: 3.9066. Min Range: -3.9066

Max Act: 3.7923. Min Act: -0.1700

Data Index: 2384651 (Open Web Text)

Max Activating Token Index: 904