Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1780 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.8022. Min Act: -0.1700

Data Index: 659505 (Open Web Text)

Max Activating Token Index: 74

Text #1

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7782. Min Act: -0.1700

Data Index: 8051950 (Open Web Text)

Max Activating Token Index: 636

Text #2

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7441. Min Act: -0.1700

Data Index: 4410712 (Open Web Text)

Max Activating Token Index: 624

Text #3

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7293. Min Act: -0.1699

Data Index: 3586548 (Open Web Text)

Max Activating Token Index: 549

Text #4

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7332. Min Act: -0.1700

Data Index: 1294296 (Open Web Text)

Max Activating Token Index: 23

Text #5

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7241. Min Act: -0.1700

Data Index: 3603365 (Open Web Text)

Max Activating Token Index: 951

Text #6

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.7146. Min Act: -0.1700

Data Index: 4160613 (Open Web Text)

Max Activating Token Index: 29

Text #7

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6881. Min Act: -0.1700

Data Index: 3836708 (Open Web Text)

Max Activating Token Index: 57

Text #8

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6817. Min Act: -0.1700

Data Index: 8010536 (Open Web Text)

Max Activating Token Index: 446

Text #9

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6750. Min Act: -0.1700

Data Index: 5157376 (Open Web Text)

Max Activating Token Index: 127

Text #10

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6765. Min Act: -0.1700

Data Index: 1903727 (Open Web Text)

Max Activating Token Index: 65

Text #11

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6554. Min Act: -0.1700

Data Index: 3637453 (Open Web Text)

Max Activating Token Index: 436

Text #12

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6439. Min Act: -0.1700

Data Index: 4160611 (Open Web Text)

Max Activating Token Index: 855

Text #13

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6553. Min Act: -0.1700

Data Index: 747506 (Open Web Text)

Max Activating Token Index: 9

Text #14

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6500. Min Act: -0.1700

Data Index: 4620416 (Open Web Text)

Max Activating Token Index: 322

Text #15

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6402. Min Act: -0.1700

Data Index: 3629233 (Open Web Text)

Max Activating Token Index: 684

Text #16

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6484. Min Act: -0.1700

Data Index: 659499 (Open Web Text)

Max Activating Token Index: 259

Text #17

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6382. Min Act: -0.1700

Data Index: 7018033 (Open Web Text)

Max Activating Token Index: 745

Text #18

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6369. Min Act: -0.1700

Data Index: 2634226 (Open Web Text)

Max Activating Token Index: 14

Text #19

Max Range: 1.8022. Min Range: -1.8022

Max Act: 1.6388. Min Act: -0.1700

Data Index: 1179283 (Open Web Text)

Max Activating Token Index: 752