Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1279 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.6296. Min Act: -0.0361

Data Index: 4755846 (Open Web Text)

Max Activating Token Index: 143

Text #1

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.6423. Min Act: -0.0572

Data Index: 1765818 (Open Web Text)

Max Activating Token Index: 105

Text #2

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.6167. Min Act: -0.0523

Data Index: 7932940 (Open Web Text)

Max Activating Token Index: 1008

Text #3

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.6156. Min Act: -0.0564

Data Index: 4957734 (Open Web Text)

Max Activating Token Index: 979

Text #4

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5922. Min Act: -0.1322

Data Index: 7317876 (Open Web Text)

Max Activating Token Index: 606

Text #5

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5902. Min Act: -0.0423

Data Index: 6341975 (Open Web Text)

Max Activating Token Index: 674

Text #6

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5907. Min Act: -0.1536

Data Index: 3227622 (Open Web Text)

Max Activating Token Index: 699

Text #7

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5895. Min Act: -0.0940

Data Index: 4436167 (Open Web Text)

Max Activating Token Index: 680

Text #8

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5880. Min Act: -0.1699

Data Index: 8096114 (Open Web Text)

Max Activating Token Index: 627

Text #9

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5871. Min Act: -0.1411

Data Index: 3618330 (Open Web Text)

Max Activating Token Index: 75

Text #10

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5862. Min Act: -0.1637

Data Index: 6675513 (Open Web Text)

Max Activating Token Index: 809

Text #11

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5853. Min Act: -0.0665

Data Index: 4185252 (Open Web Text)

Max Activating Token Index: 799

Text #12

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5836. Min Act: -0.1154

Data Index: 7778183 (Open Web Text)

Max Activating Token Index: 730

Text #13

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5799. Min Act: -0.1237

Data Index: 4496803 (Open Web Text)

Max Activating Token Index: 970

Text #14

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5853. Min Act: -0.0011

Data Index: 6439971 (Open Web Text)

Max Activating Token Index: 771

Text #15

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5678. Min Act: -0.0301

Data Index: 1305838 (Open Web Text)

Max Activating Token Index: 894

Text #16

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5676. Min Act: -0.1221

Data Index: 769995 (Open Web Text)

Max Activating Token Index: 587

Text #17

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5766. Min Act: -0.1125

Data Index: 6438714 (Open Web Text)

Max Activating Token Index: 977

Text #18

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5754. Min Act: -0.1096

Data Index: 1344839 (Open Web Text)

Max Activating Token Index: 872

Text #19

Max Range: 1.6423. Min Range: -1.6423

Max Act: 1.5729. Min Act: -0.0461

Data Index: 3386500 (Open Web Text)

Max Activating Token Index: 824