Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 493 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1493. Min Range: -5.1493

Max Act: 5.1493. Min Act: -0.1700

Data Index: 824321 (Open Web Text)

Max Activating Token Index: 258

Text #1

Max Range: 5.1493. Min Range: -5.1493

Max Act: 5.1288. Min Act: -0.1700

Data Index: 824328 (Open Web Text)

Max Activating Token Index: 450

Text #2

Max Range: 5.1493. Min Range: -5.1493

Max Act: 5.0560. Min Act: -0.1700

Data Index: 5998536 (Open Web Text)

Max Activating Token Index: 806

Text #3

Max Range: 5.1493. Min Range: -5.1493

Max Act: 5.0356. Min Act: -0.1700

Data Index: 6729595 (Open Web Text)

Max Activating Token Index: 95

Text #4

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.9418. Min Act: -0.1700

Data Index: 8578131 (Open Web Text)

Max Activating Token Index: 264

Text #5

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.9359. Min Act: -0.1700

Data Index: 1317482 (Open Web Text)

Max Activating Token Index: 191

Text #6

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.9708. Min Act: -0.1700

Data Index: 4879356 (Open Web Text)

Max Activating Token Index: 166

Text #7

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.9596. Min Act: -0.1700

Data Index: 5078460 (Open Web Text)

Max Activating Token Index: 368

Text #8

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.9520. Min Act: -0.1700

Data Index: 1223973 (Open Web Text)

Max Activating Token Index: 553

Text #9

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8729. Min Act: -0.1700

Data Index: 3275203 (Open Web Text)

Max Activating Token Index: 194

Text #10

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8859. Min Act: -0.1700

Data Index: 8765414 (Open Web Text)

Max Activating Token Index: 619

Text #11

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8425. Min Act: -0.1700

Data Index: 7250495 (Open Web Text)

Max Activating Token Index: 257

Text #12

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8951. Min Act: -0.1700

Data Index: 1775128 (Open Web Text)

Max Activating Token Index: 530

Text #13

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8509. Min Act: -0.1700

Data Index: 5470412 (Open Web Text)

Max Activating Token Index: 348

Text #14

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8604. Min Act: -0.1700

Data Index: 3897093 (Open Web Text)

Max Activating Token Index: 976

Text #15

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8649. Min Act: -0.1700

Data Index: 1710321 (Open Web Text)

Max Activating Token Index: 129

Text #16

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8833. Min Act: -0.1700

Data Index: 6078313 (Open Web Text)

Max Activating Token Index: 995

Text #17

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8613. Min Act: -0.1700

Data Index: 7315223 (Open Web Text)

Max Activating Token Index: 127

Text #18

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8486. Min Act: -0.1700

Data Index: 213391 (Open Web Text)

Max Activating Token Index: 262

Text #19

Max Range: 5.1493. Min Range: -5.1493

Max Act: 4.8210. Min Act: -0.1700

Data Index: 794458 (Open Web Text)

Max Activating Token Index: 960