Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1640 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.9803. Min Act: -0.1163

Data Index: 4757051 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.9393. Min Act: -0.1552

Data Index: 3666529 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.9131. Min Act: -0.1509

Data Index: 543879 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.9155. Min Act: -0.1376

Data Index: 2342282 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8848. Min Act: -0.1588

Data Index: 5785904 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8889. Min Act: -0.1362

Data Index: 3827779 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8816. Min Act: -0.1658

Data Index: 4069245 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8609. Min Act: -0.1543

Data Index: 1062038 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8572. Min Act: -0.1588

Data Index: 7240607 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8574. Min Act: -0.1585

Data Index: 4201817 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8529. Min Act: -0.1691

Data Index: 3304230 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8325. Min Act: -0.1306

Data Index: 5883439 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8376. Min Act: -0.1086

Data Index: 5907974 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8518. Min Act: -0.1459

Data Index: 752929 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8337. Min Act: -0.1369

Data Index: 6063904 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8464. Min Act: -0.1651

Data Index: 2460809 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8439. Min Act: -0.1471

Data Index: 4585953 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8268. Min Act: -0.1404

Data Index: 6716362 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8435. Min Act: -0.1402

Data Index: 2302586 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 1.9803. Min Range: -1.9803

Max Act: 1.8197. Min Act: -0.1065

Data Index: 5632166 (Open Web Text)

Max Activating Token Index: 1023