Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2743 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.2067. Min Act: -0.1700

Data Index: 1719649 (Open Web Text)

Max Activating Token Index: 496

Text #1

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.2062. Min Act: -0.1700

Data Index: 5809273 (Open Web Text)

Max Activating Token Index: 305

Text #2

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1676. Min Act: -0.1700

Data Index: 6635685 (Open Web Text)

Max Activating Token Index: 826

Text #3

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1281. Min Act: -0.1700

Data Index: 6702022 (Open Web Text)

Max Activating Token Index: 749

Text #4

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1237. Min Act: -0.1700

Data Index: 6170440 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1369. Min Act: -0.1700

Data Index: 5838170 (Open Web Text)

Max Activating Token Index: 101

Text #6

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1411. Min Act: -0.1700

Data Index: 4779995 (Open Web Text)

Max Activating Token Index: 443

Text #7

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0768. Min Act: -0.1700

Data Index: 2065705 (Open Web Text)

Max Activating Token Index: 31

Text #8

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0821. Min Act: -0.1700

Data Index: 3263076 (Open Web Text)

Max Activating Token Index: 444

Text #9

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0801. Min Act: -0.1700

Data Index: 6395115 (Open Web Text)

Max Activating Token Index: 117

Text #10

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0860. Min Act: -0.1700

Data Index: 6601157 (Open Web Text)

Max Activating Token Index: 323

Text #11

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.1128. Min Act: -0.1700

Data Index: 3063616 (Open Web Text)

Max Activating Token Index: 740

Text #12

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0604. Min Act: -0.1700

Data Index: 4779994 (Open Web Text)

Max Activating Token Index: 851

Text #13

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0412. Min Act: -0.1700

Data Index: 5072809 (Open Web Text)

Max Activating Token Index: 493

Text #14

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0770. Min Act: -0.1700

Data Index: 4438686 (Open Web Text)

Max Activating Token Index: 449

Text #15

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0546. Min Act: -0.1700

Data Index: 6257636 (Open Web Text)

Max Activating Token Index: 318

Text #16

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0361. Min Act: -0.1700

Data Index: 2854097 (Open Web Text)

Max Activating Token Index: 659

Text #17

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0716. Min Act: -0.1700

Data Index: 5379737 (Open Web Text)

Max Activating Token Index: 997

Text #18

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0684. Min Act: -0.1700

Data Index: 1468737 (Open Web Text)

Max Activating Token Index: 299

Text #19

Max Range: 4.2067. Min Range: -4.2067

Max Act: 4.0530. Min Act: -0.1700

Data Index: 1671867 (Open Web Text)

Max Activating Token Index: 321