Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 359 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.1766. Min Act: -0.1700

Data Index: 6901589 (Open Web Text)

Max Activating Token Index: 579

Text #1

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.1437. Min Act: -0.1700

Data Index: 8319787 (Open Web Text)

Max Activating Token Index: 377

Text #2

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.1115. Min Act: -0.1700

Data Index: 4331698 (Open Web Text)

Max Activating Token Index: 126

Text #3

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0828. Min Act: -0.1700

Data Index: 7567934 (Open Web Text)

Max Activating Token Index: 719

Text #4

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0712. Min Act: -0.1700

Data Index: 2917809 (Open Web Text)

Max Activating Token Index: 967

Text #5

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0699. Min Act: -0.1700

Data Index: 2169354 (Open Web Text)

Max Activating Token Index: 337

Text #6

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0699. Min Act: -0.1700

Data Index: 7523011 (Open Web Text)

Max Activating Token Index: 486

Text #7

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0647. Min Act: -0.1700

Data Index: 2405667 (Open Web Text)

Max Activating Token Index: 495

Text #8

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0642. Min Act: -0.1700

Data Index: 6058357 (Open Web Text)

Max Activating Token Index: 675

Text #9

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0564. Min Act: -0.1700

Data Index: 7295443 (Open Web Text)

Max Activating Token Index: 714

Text #10

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0703. Min Act: -0.1700

Data Index: 5481869 (Open Web Text)

Max Activating Token Index: 702

Text #11

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0486. Min Act: -0.1700

Data Index: 5431811 (Open Web Text)

Max Activating Token Index: 592

Text #12

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0513. Min Act: -0.1700

Data Index: 8287101 (Open Web Text)

Max Activating Token Index: 588

Text #13

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0313. Min Act: -0.1700

Data Index: 7318010 (Open Web Text)

Max Activating Token Index: 970

Text #14

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0285. Min Act: -0.1700

Data Index: 7247060 (Open Web Text)

Max Activating Token Index: 97

Text #15

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0478. Min Act: -0.1700

Data Index: 3470206 (Open Web Text)

Max Activating Token Index: 463

Text #16

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0394. Min Act: -0.1700

Data Index: 4255458 (Open Web Text)

Max Activating Token Index: 494

Text #17

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0447. Min Act: -0.1700

Data Index: 6370292 (Open Web Text)

Max Activating Token Index: 785

Text #18

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0338. Min Act: -0.1700

Data Index: 2962482 (Open Web Text)

Max Activating Token Index: 669

Text #19

Max Range: 3.1766. Min Range: -3.1766

Max Act: 3.0224. Min Act: -0.1700

Data Index: 217851 (Open Web Text)

Max Activating Token Index: 806