Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 524 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.2317. Min Act: -0.1700

Data Index: 1846572 (Open Web Text)

Max Activating Token Index: 427

Text #1

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.1459. Min Act: -0.1700

Data Index: 7439913 (Open Web Text)

Max Activating Token Index: 315

Text #2

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.1223. Min Act: -0.1700

Data Index: 6628545 (Open Web Text)

Max Activating Token Index: 785

Text #3

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.1235. Min Act: -0.1700

Data Index: 4029667 (Open Web Text)

Max Activating Token Index: 661

Text #4

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0873. Min Act: -0.1700

Data Index: 3633960 (Open Web Text)

Max Activating Token Index: 603

Text #5

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0831. Min Act: -0.1700

Data Index: 8143778 (Open Web Text)

Max Activating Token Index: 70

Text #6

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0536. Min Act: -0.1700

Data Index: 5136750 (Open Web Text)

Max Activating Token Index: 636

Text #7

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0769. Min Act: -0.1700

Data Index: 8285258 (Open Web Text)

Max Activating Token Index: 747

Text #8

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0548. Min Act: -0.1700

Data Index: 4704670 (Open Web Text)

Max Activating Token Index: 191

Text #9

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0542. Min Act: -0.1700

Data Index: 3810140 (Open Web Text)

Max Activating Token Index: 734

Text #10

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0681. Min Act: -0.1700

Data Index: 4998593 (Open Web Text)

Max Activating Token Index: 624

Text #11

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0471. Min Act: -0.1700

Data Index: 3112557 (Open Web Text)

Max Activating Token Index: 188

Text #12

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0636. Min Act: -0.1700

Data Index: 4451953 (Open Web Text)

Max Activating Token Index: 593

Text #13

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0587. Min Act: -0.1700

Data Index: 3518600 (Open Web Text)

Max Activating Token Index: 831

Text #14

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0443. Min Act: -0.1700

Data Index: 3595033 (Open Web Text)

Max Activating Token Index: 767

Text #15

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0412. Min Act: -0.1700

Data Index: 6623832 (Open Web Text)

Max Activating Token Index: 306

Text #16

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0401. Min Act: -0.1700

Data Index: 6114221 (Open Web Text)

Max Activating Token Index: 179

Text #17

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0335. Min Act: -0.1700

Data Index: 4182693 (Open Web Text)

Max Activating Token Index: 335

Text #18

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0036. Min Act: -0.1700

Data Index: 68125 (Open Web Text)

Max Activating Token Index: 543

Text #19

Max Range: 2.2317. Min Range: -2.2317

Max Act: 2.0162. Min Act: -0.1700

Data Index: 1973831 (Open Web Text)

Max Activating Token Index: 703