Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2951 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.3651. Min Act: -0.1700

Data Index: 5905118 (Open Web Text)

Max Activating Token Index: 693

Text #1

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.2251. Min Act: -0.1700

Data Index: 5832997 (Open Web Text)

Max Activating Token Index: 760

Text #2

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.2227. Min Act: -0.1700

Data Index: 3279280 (Open Web Text)

Max Activating Token Index: 589

Text #3

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.2183. Min Act: -0.1700

Data Index: 3424657 (Open Web Text)

Max Activating Token Index: 626

Text #4

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1916. Min Act: -0.1700

Data Index: 5607059 (Open Web Text)

Max Activating Token Index: 403

Text #5

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1621. Min Act: -0.1700

Data Index: 5430894 (Open Web Text)

Max Activating Token Index: 678

Text #6

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1867. Min Act: -0.1700

Data Index: 8631818 (Open Web Text)

Max Activating Token Index: 790

Text #7

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1623. Min Act: -0.1700

Data Index: 3746696 (Open Web Text)

Max Activating Token Index: 827

Text #8

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1528. Min Act: -0.1700

Data Index: 885827 (Open Web Text)

Max Activating Token Index: 909

Text #9

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1820. Min Act: -0.1700

Data Index: 5415834 (Open Web Text)

Max Activating Token Index: 595

Text #10

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1710. Min Act: -0.1700

Data Index: 4061584 (Open Web Text)

Max Activating Token Index: 694

Text #11

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1586. Min Act: -0.1700

Data Index: 7074606 (Open Web Text)

Max Activating Token Index: 40

Text #12

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1458. Min Act: -0.1700

Data Index: 2579702 (Open Web Text)

Max Activating Token Index: 554

Text #13

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1224. Min Act: -0.1700

Data Index: 565197 (Open Web Text)

Max Activating Token Index: 660

Text #14

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1402. Min Act: -0.1700

Data Index: 7004457 (Open Web Text)

Max Activating Token Index: 523

Text #15

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1457. Min Act: -0.1700

Data Index: 4954416 (Open Web Text)

Max Activating Token Index: 272

Text #16

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1341. Min Act: -0.1700

Data Index: 3858717 (Open Web Text)

Max Activating Token Index: 626

Text #17

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1352. Min Act: -0.1700

Data Index: 3490941 (Open Web Text)

Max Activating Token Index: 285

Text #18

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1463. Min Act: -0.1700

Data Index: 8306009 (Open Web Text)

Max Activating Token Index: 443

Text #19

Max Range: 3.3651. Min Range: -3.3651

Max Act: 3.1360. Min Act: -0.1700

Data Index: 2941832 (Open Web Text)

Max Activating Token Index: 140