Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 21 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.8308. Min Act: -0.1700

Data Index: 3469164 (Open Web Text)

Max Activating Token Index: 978

Text #1

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.8224. Min Act: -0.1700

Data Index: 5545658 (Open Web Text)

Max Activating Token Index: 214

Text #2

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.7826. Min Act: -0.1700

Data Index: 8492493 (Open Web Text)

Max Activating Token Index: 515

Text #3

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.7603. Min Act: -0.1700

Data Index: 8488618 (Open Web Text)

Max Activating Token Index: 596

Text #4

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.7550. Min Act: -0.1700

Data Index: 5835735 (Open Web Text)

Max Activating Token Index: 712

Text #5

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.7705. Min Act: -0.1700

Data Index: 6387640 (Open Web Text)

Max Activating Token Index: 615

Text #6

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6968. Min Act: -0.1700

Data Index: 6585033 (Open Web Text)

Max Activating Token Index: 106

Text #7

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6612. Min Act: -0.1700

Data Index: 8692037 (Open Web Text)

Max Activating Token Index: 806

Text #8

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.7032. Min Act: -0.1700

Data Index: 4156335 (Open Web Text)

Max Activating Token Index: 475

Text #9

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6372. Min Act: -0.1700

Data Index: 6712099 (Open Web Text)

Max Activating Token Index: 288

Text #10

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6795. Min Act: -0.1700

Data Index: 762063 (Open Web Text)

Max Activating Token Index: 612

Text #11

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6357. Min Act: -0.1700

Data Index: 7188722 (Open Web Text)

Max Activating Token Index: 705

Text #12

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6445. Min Act: -0.1700

Data Index: 6108902 (Open Web Text)

Max Activating Token Index: 649

Text #13

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6674. Min Act: -0.1700

Data Index: 2093341 (Open Web Text)

Max Activating Token Index: 759

Text #14

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6336. Min Act: -0.1700

Data Index: 7579543 (Open Web Text)

Max Activating Token Index: 717

Text #15

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6320. Min Act: -0.1700

Data Index: 100589 (Open Web Text)

Max Activating Token Index: 607

Text #16

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6310. Min Act: -0.1700

Data Index: 6921294 (Open Web Text)

Max Activating Token Index: 974

Text #17

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6032. Min Act: -0.1700

Data Index: 1610075 (Open Web Text)

Max Activating Token Index: 280

Text #18

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6144. Min Act: -0.1700

Data Index: 6043710 (Open Web Text)

Max Activating Token Index: 696

Text #19

Max Range: 3.8308. Min Range: -3.8308

Max Act: 3.6131. Min Act: -0.1700

Data Index: 3813169 (Open Web Text)

Max Activating Token Index: 398