Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2759 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.9050. Min Act: -0.1700

Data Index: 4121182 (Open Web Text)

Max Activating Token Index: 387

Text #1

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.9019. Min Act: -0.1700

Data Index: 8482167 (Open Web Text)

Max Activating Token Index: 389

Text #2

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8668. Min Act: -0.1700

Data Index: 2110698 (Open Web Text)

Max Activating Token Index: 471

Text #3

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8782. Min Act: -0.1700

Data Index: 5849227 (Open Web Text)

Max Activating Token Index: 447

Text #4

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8920. Min Act: -0.1700

Data Index: 4749362 (Open Web Text)

Max Activating Token Index: 339

Text #5

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8857. Min Act: -0.1700

Data Index: 669731 (Open Web Text)

Max Activating Token Index: 373

Text #6

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8625. Min Act: -0.1700

Data Index: 6612405 (Open Web Text)

Max Activating Token Index: 396

Text #7

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8457. Min Act: -0.1700

Data Index: 636493 (Open Web Text)

Max Activating Token Index: 445

Text #8

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8390. Min Act: -0.1700

Data Index: 2638164 (Open Web Text)

Max Activating Token Index: 416

Text #9

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8536. Min Act: -0.1700

Data Index: 5615977 (Open Web Text)

Max Activating Token Index: 464

Text #10

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8477. Min Act: -0.1700

Data Index: 5928634 (Open Web Text)

Max Activating Token Index: 454

Text #11

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8456. Min Act: -0.1700

Data Index: 3598803 (Open Web Text)

Max Activating Token Index: 471

Text #12

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8602. Min Act: -0.1700

Data Index: 2817829 (Open Web Text)

Max Activating Token Index: 460

Text #13

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8163. Min Act: -0.1700

Data Index: 1997853 (Open Web Text)

Max Activating Token Index: 448

Text #14

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8395. Min Act: -0.1700

Data Index: 6422710 (Open Web Text)

Max Activating Token Index: 425

Text #15

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8418. Min Act: -0.1700

Data Index: 653563 (Open Web Text)

Max Activating Token Index: 426

Text #16

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8345. Min Act: -0.1700

Data Index: 1561793 (Open Web Text)

Max Activating Token Index: 453

Text #17

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8325. Min Act: -0.1700

Data Index: 340422 (Open Web Text)

Max Activating Token Index: 422

Text #18

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8349. Min Act: -0.1700

Data Index: 5169783 (Open Web Text)

Max Activating Token Index: 471

Text #19

Max Range: 0.9050. Min Range: -0.9050

Max Act: 0.8412. Min Act: -0.1700

Data Index: 444534 (Open Web Text)

Max Activating Token Index: 432