Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1019 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.4842. Min Act: -0.1700

Data Index: 1204596 (Open Web Text)

Max Activating Token Index: 502

Text #1

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.4652. Min Act: -0.1700

Data Index: 3856656 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.4590. Min Act: -0.1700

Data Index: 2337640 (Open Web Text)

Max Activating Token Index: 413

Text #3

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.4493. Min Act: -0.1700

Data Index: 1335078 (Open Web Text)

Max Activating Token Index: 503

Text #4

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3945. Min Act: -0.1700

Data Index: 910436 (Open Web Text)

Max Activating Token Index: 538

Text #5

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3730. Min Act: -0.1700

Data Index: 6016465 (Open Web Text)

Max Activating Token Index: 850

Text #6

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3276. Min Act: -0.1700

Data Index: 2246994 (Open Web Text)

Max Activating Token Index: 571

Text #7

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3217. Min Act: -0.1700

Data Index: 3998374 (Open Web Text)

Max Activating Token Index: 611

Text #8

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3291. Min Act: -0.1700

Data Index: 341717 (Open Web Text)

Max Activating Token Index: 486

Text #9

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3156. Min Act: -0.1700

Data Index: 3268310 (Open Web Text)

Max Activating Token Index: 638

Text #10

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3256. Min Act: -0.1700

Data Index: 7617312 (Open Web Text)

Max Activating Token Index: 502

Text #11

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3345. Min Act: -0.1700

Data Index: 7042744 (Open Web Text)

Max Activating Token Index: 559

Text #12

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3328. Min Act: -0.1700

Data Index: 4752892 (Open Web Text)

Max Activating Token Index: 527

Text #13

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.2974. Min Act: -0.1700

Data Index: 42572 (Open Web Text)

Max Activating Token Index: 461

Text #14

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.2824. Min Act: -0.1700

Data Index: 6449883 (Open Web Text)

Max Activating Token Index: 653

Text #15

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.2984. Min Act: -0.1700

Data Index: 2530253 (Open Web Text)

Max Activating Token Index: 506

Text #16

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.2931. Min Act: -0.1700

Data Index: 6353463 (Open Web Text)

Max Activating Token Index: 422

Text #17

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3149. Min Act: -0.1700

Data Index: 6269561 (Open Web Text)

Max Activating Token Index: 627

Text #18

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3100. Min Act: -0.1700

Data Index: 848212 (Open Web Text)

Max Activating Token Index: 831

Text #19

Max Range: 3.4842. Min Range: -3.4842

Max Act: 3.3051. Min Act: -0.1700

Data Index: 100907 (Open Web Text)

Max Activating Token Index: 467