Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2517 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.6279. Min Act: -0.1700

Data Index: 3002403 (Open Web Text)

Max Activating Token Index: 489

Text #1

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.5749. Min Act: -0.1700

Data Index: 3452523 (Open Web Text)

Max Activating Token Index: 292

Text #2

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.5521. Min Act: -0.1700

Data Index: 5245117 (Open Web Text)

Max Activating Token Index: 722

Text #3

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.5324. Min Act: -0.1700

Data Index: 6793761 (Open Web Text)

Max Activating Token Index: 363

Text #4

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.4561. Min Act: -0.1700

Data Index: 7029060 (Open Web Text)

Max Activating Token Index: 997

Text #5

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.4439. Min Act: -0.1700

Data Index: 3099209 (Open Web Text)

Max Activating Token Index: 367

Text #6

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.4115. Min Act: -0.1700

Data Index: 1704147 (Open Web Text)

Max Activating Token Index: 1007

Text #7

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3838. Min Act: -0.1700

Data Index: 4301490 (Open Web Text)

Max Activating Token Index: 305

Text #8

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3889. Min Act: -0.1700

Data Index: 7170810 (Open Web Text)

Max Activating Token Index: 264

Text #9

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3915. Min Act: -0.1700

Data Index: 6793762 (Open Web Text)

Max Activating Token Index: 322

Text #10

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.4031. Min Act: -0.1700

Data Index: 156222 (Open Web Text)

Max Activating Token Index: 239

Text #11

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.4145. Min Act: -0.1700

Data Index: 7981666 (Open Web Text)

Max Activating Token Index: 964

Text #12

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3824. Min Act: -0.1700

Data Index: 463617 (Open Web Text)

Max Activating Token Index: 687

Text #13

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3968. Min Act: -0.1700

Data Index: 931633 (Open Web Text)

Max Activating Token Index: 162

Text #14

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3883. Min Act: -0.1700

Data Index: 7250379 (Open Web Text)

Max Activating Token Index: 518

Text #15

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3748. Min Act: -0.1700

Data Index: 6333821 (Open Web Text)

Max Activating Token Index: 608

Text #16

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3490. Min Act: -0.1700

Data Index: 6426737 (Open Web Text)

Max Activating Token Index: 366

Text #17

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3298. Min Act: -0.1700

Data Index: 6144458 (Open Web Text)

Max Activating Token Index: 194

Text #18

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3609. Min Act: -0.1700

Data Index: 1531374 (Open Web Text)

Max Activating Token Index: 734

Text #19

Max Range: 3.6279. Min Range: -3.6279

Max Act: 3.3654. Min Act: -0.1700

Data Index: 3657137 (Open Web Text)

Max Activating Token Index: 778