Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3010 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.7228. Min Act: -0.1700

Data Index: 4050994 (Open Web Text)

Max Activating Token Index: 898

Text #1

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6954. Min Act: -0.1700

Data Index: 343700 (Open Web Text)

Max Activating Token Index: 389

Text #2

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6935. Min Act: -0.1700

Data Index: 6167945 (Open Web Text)

Max Activating Token Index: 338

Text #3

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6974. Min Act: -0.1700

Data Index: 7923346 (Open Web Text)

Max Activating Token Index: 399

Text #4

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6836. Min Act: -0.1700

Data Index: 5697729 (Open Web Text)

Max Activating Token Index: 721

Text #5

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6687. Min Act: -0.1700

Data Index: 4272952 (Open Web Text)

Max Activating Token Index: 735

Text #6

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6758. Min Act: -0.1700

Data Index: 4943650 (Open Web Text)

Max Activating Token Index: 455

Text #7

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6717. Min Act: -0.1700

Data Index: 3343144 (Open Web Text)

Max Activating Token Index: 856

Text #8

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6768. Min Act: -0.1700

Data Index: 595016 (Open Web Text)

Max Activating Token Index: 717

Text #9

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6685. Min Act: -0.1700

Data Index: 8482015 (Open Web Text)

Max Activating Token Index: 859

Text #10

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6616. Min Act: -0.1700

Data Index: 7502139 (Open Web Text)

Max Activating Token Index: 882

Text #11

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6619. Min Act: -0.1700

Data Index: 5399881 (Open Web Text)

Max Activating Token Index: 67

Text #12

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6650. Min Act: -0.1700

Data Index: 1230462 (Open Web Text)

Max Activating Token Index: 194

Text #13

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6645. Min Act: -0.1700

Data Index: 631968 (Open Web Text)

Max Activating Token Index: 750

Text #14

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6587. Min Act: -0.1700

Data Index: 8718644 (Open Web Text)

Max Activating Token Index: 152

Text #15

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6516. Min Act: -0.1700

Data Index: 5858175 (Open Web Text)

Max Activating Token Index: 833

Text #16

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6558. Min Act: -0.1700

Data Index: 8826534 (Open Web Text)

Max Activating Token Index: 420

Text #17

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6627. Min Act: -0.1700

Data Index: 1667290 (Open Web Text)

Max Activating Token Index: 355

Text #18

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6617. Min Act: -0.1700

Data Index: 451773 (Open Web Text)

Max Activating Token Index: 849

Text #19

Max Range: 1.7228. Min Range: -1.7228

Max Act: 1.6629. Min Act: -0.1700

Data Index: 552382 (Open Web Text)

Max Activating Token Index: 605