Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2656 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1423. Min Act: -0.1700

Data Index: 1789147 (Open Web Text)

Max Activating Token Index: 527

Text #1

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1541. Min Act: -0.1700

Data Index: 3930406 (Open Web Text)

Max Activating Token Index: 721

Text #2

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1625. Min Act: -0.1700

Data Index: 1322 (Open Web Text)

Max Activating Token Index: 880

Text #3

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1629. Min Act: -0.1700

Data Index: 6533919 (Open Web Text)

Max Activating Token Index: 329

Text #4

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1259. Min Act: -0.1700

Data Index: 7946927 (Open Web Text)

Max Activating Token Index: 944

Text #5

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1151. Min Act: -0.1700

Data Index: 3869665 (Open Web Text)

Max Activating Token Index: 267

Text #6

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1071. Min Act: -0.1700

Data Index: 87482 (Open Web Text)

Max Activating Token Index: 279

Text #7

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1046. Min Act: -0.1700

Data Index: 5735784 (Open Web Text)

Max Activating Token Index: 957

Text #8

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0508. Min Act: -0.1700

Data Index: 4643268 (Open Web Text)

Max Activating Token Index: 551

Text #9

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0728. Min Act: -0.1700

Data Index: 3871749 (Open Web Text)

Max Activating Token Index: 575

Text #10

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.1109. Min Act: -0.1700

Data Index: 3721885 (Open Web Text)

Max Activating Token Index: 380

Text #11

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0648. Min Act: -0.1700

Data Index: 1987785 (Open Web Text)

Max Activating Token Index: 775

Text #12

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0708. Min Act: -0.1700

Data Index: 7030307 (Open Web Text)

Max Activating Token Index: 680

Text #13

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0831. Min Act: -0.1700

Data Index: 6455774 (Open Web Text)

Max Activating Token Index: 849

Text #14

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0691. Min Act: -0.1700

Data Index: 3143619 (Open Web Text)

Max Activating Token Index: 670

Text #15

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0480. Min Act: -0.1700

Data Index: 4638777 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0511. Min Act: -0.1700

Data Index: 5184722 (Open Web Text)

Max Activating Token Index: 287

Text #17

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0568. Min Act: -0.1700

Data Index: 5639950 (Open Web Text)

Max Activating Token Index: 383

Text #18

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0580. Min Act: -0.1700

Data Index: 3917708 (Open Web Text)

Max Activating Token Index: 460

Text #19

Max Range: 3.1629. Min Range: -3.1629

Max Act: 3.0490. Min Act: -0.1700

Data Index: 2844379 (Open Web Text)

Max Activating Token Index: 482