Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 404 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1662. Min Range: -5.1662

Max Act: 5.1662. Min Act: -0.1700

Data Index: 2601933 (Open Web Text)

Max Activating Token Index: 596

Text #1

Max Range: 5.1662. Min Range: -5.1662

Max Act: 5.1285. Min Act: -0.1700

Data Index: 823833 (Open Web Text)

Max Activating Token Index: 176

Text #2

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9994. Min Act: -0.1700

Data Index: 6110757 (Open Web Text)

Max Activating Token Index: 185

Text #3

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9871. Min Act: -0.1700

Data Index: 4676356 (Open Web Text)

Max Activating Token Index: 364

Text #4

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9869. Min Act: -0.1700

Data Index: 7468340 (Open Web Text)

Max Activating Token Index: 726

Text #5

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9463. Min Act: -0.1700

Data Index: 5393756 (Open Web Text)

Max Activating Token Index: 579

Text #6

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9287. Min Act: -0.1700

Data Index: 2116236 (Open Web Text)

Max Activating Token Index: 80

Text #7

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9329. Min Act: -0.1700

Data Index: 3069090 (Open Web Text)

Max Activating Token Index: 480

Text #8

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9085. Min Act: -0.1700

Data Index: 8595332 (Open Web Text)

Max Activating Token Index: 223

Text #9

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9084. Min Act: -0.1700

Data Index: 203338 (Open Web Text)

Max Activating Token Index: 565

Text #10

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9022. Min Act: -0.1700

Data Index: 8676643 (Open Web Text)

Max Activating Token Index: 335

Text #11

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8909. Min Act: -0.1700

Data Index: 6232009 (Open Web Text)

Max Activating Token Index: 108

Text #12

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8829. Min Act: -0.1700

Data Index: 2714505 (Open Web Text)

Max Activating Token Index: 996

Text #13

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.9066. Min Act: -0.1700

Data Index: 1831548 (Open Web Text)

Max Activating Token Index: 357

Text #14

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8892. Min Act: -0.1700

Data Index: 7548292 (Open Web Text)

Max Activating Token Index: 932

Text #15

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8889. Min Act: -0.1700

Data Index: 8335246 (Open Web Text)

Max Activating Token Index: 475

Text #16

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8753. Min Act: -0.1700

Data Index: 3856784 (Open Web Text)

Max Activating Token Index: 536

Text #17

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8876. Min Act: -0.1700

Data Index: 5327505 (Open Web Text)

Max Activating Token Index: 221

Text #18

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8569. Min Act: -0.1700

Data Index: 7692293 (Open Web Text)

Max Activating Token Index: 192

Text #19

Max Range: 5.1662. Min Range: -5.1662

Max Act: 4.8831. Min Act: -0.1700

Data Index: 3529776 (Open Web Text)

Max Activating Token Index: 523