Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 117 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1846. Min Range: -5.1846

Max Act: 5.1846. Min Act: -0.1700

Data Index: 807180 (Open Web Text)

Max Activating Token Index: 467

Text #1

Max Range: 5.1846. Min Range: -5.1846

Max Act: 5.0529. Min Act: -0.1700

Data Index: 8072646 (Open Web Text)

Max Activating Token Index: 391

Text #2

Max Range: 5.1846. Min Range: -5.1846

Max Act: 5.0110. Min Act: -0.1700

Data Index: 8072644 (Open Web Text)

Max Activating Token Index: 814

Text #3

Max Range: 5.1846. Min Range: -5.1846

Max Act: 5.0133. Min Act: -0.1700

Data Index: 5860557 (Open Web Text)

Max Activating Token Index: 132

Text #4

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.9734. Min Act: -0.1700

Data Index: 6130767 (Open Web Text)

Max Activating Token Index: 693

Text #5

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.9415. Min Act: -0.1700

Data Index: 8552845 (Open Web Text)

Max Activating Token Index: 468

Text #6

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.9376. Min Act: -0.1700

Data Index: 4017123 (Open Web Text)

Max Activating Token Index: 286

Text #7

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.9449. Min Act: -0.1700

Data Index: 2182261 (Open Web Text)

Max Activating Token Index: 760

Text #8

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8616. Min Act: -0.1700

Data Index: 224224 (Open Web Text)

Max Activating Token Index: 346

Text #9

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8612. Min Act: -0.1700

Data Index: 556552 (Open Web Text)

Max Activating Token Index: 677

Text #10

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8806. Min Act: -0.1700

Data Index: 3110617 (Open Web Text)

Max Activating Token Index: 568

Text #11

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8644. Min Act: -0.1700

Data Index: 1616924 (Open Web Text)

Max Activating Token Index: 54

Text #12

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8644. Min Act: -0.1700

Data Index: 4653263 (Open Web Text)

Max Activating Token Index: 85

Text #13

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8454. Min Act: -0.1700

Data Index: 2051664 (Open Web Text)

Max Activating Token Index: 33

Text #14

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8537. Min Act: -0.1700

Data Index: 5233932 (Open Web Text)

Max Activating Token Index: 632

Text #15

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8497. Min Act: -0.1700

Data Index: 502961 (Open Web Text)

Max Activating Token Index: 387

Text #16

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8288. Min Act: -0.1700

Data Index: 3419098 (Open Web Text)

Max Activating Token Index: 316

Text #17

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8512. Min Act: -0.1700

Data Index: 4713619 (Open Web Text)

Max Activating Token Index: 580

Text #18

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.7747. Min Act: -0.1700

Data Index: 6210127 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 5.1846. Min Range: -5.1846

Max Act: 4.8002. Min Act: -0.1700

Data Index: 2983302 (Open Web Text)

Max Activating Token Index: 608