Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1225 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0253. Min Act: -0.1699

Data Index: 5637566 (Open Web Text)

Max Activating Token Index: 721

Text #1

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0290. Min Act: -0.1638

Data Index: 8042763 (Open Web Text)

Max Activating Token Index: 375

Text #2

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0176. Min Act: -0.1403

Data Index: 4106780 (Open Web Text)

Max Activating Token Index: 920

Text #3

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0192. Min Act: -0.1659

Data Index: 4267081 (Open Web Text)

Max Activating Token Index: 569

Text #4

Max Range: 1.0290. Min Range: -1.0290

Max Act: 1.0125. Min Act: -0.1695

Data Index: 4672067 (Open Web Text)

Max Activating Token Index: 244

Text #5

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9914. Min Act: -0.1608

Data Index: 799843 (Open Web Text)

Max Activating Token Index: 795

Text #6

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9817. Min Act: -0.1655

Data Index: 5278120 (Open Web Text)

Max Activating Token Index: 396

Text #7

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9784. Min Act: -0.1648

Data Index: 6815935 (Open Web Text)

Max Activating Token Index: 391

Text #8

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9811. Min Act: -0.1696

Data Index: 5512266 (Open Web Text)

Max Activating Token Index: 237

Text #9

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9805. Min Act: -0.1673

Data Index: 5276577 (Open Web Text)

Max Activating Token Index: 947

Text #10

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9715. Min Act: -0.1590

Data Index: 7950487 (Open Web Text)

Max Activating Token Index: 532

Text #11

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9730. Min Act: -0.1688

Data Index: 7213251 (Open Web Text)

Max Activating Token Index: 965

Text #12

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9698. Min Act: -0.1700

Data Index: 2343030 (Open Web Text)

Max Activating Token Index: 892

Text #13

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9740. Min Act: -0.1695

Data Index: 4498338 (Open Web Text)

Max Activating Token Index: 362

Text #14

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9634. Min Act: -0.1665

Data Index: 5934145 (Open Web Text)

Max Activating Token Index: 552

Text #15

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9667. Min Act: -0.1618

Data Index: 7360345 (Open Web Text)

Max Activating Token Index: 161

Text #16

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9679. Min Act: -0.1587

Data Index: 3453938 (Open Web Text)

Max Activating Token Index: 925

Text #17

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9653. Min Act: -0.1651

Data Index: 811268 (Open Web Text)

Max Activating Token Index: 395

Text #18

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9615. Min Act: -0.1523

Data Index: 7674745 (Open Web Text)

Max Activating Token Index: 894

Text #19

Max Range: 1.0290. Min Range: -1.0290

Max Act: 0.9624. Min Act: -0.1691

Data Index: 4045633 (Open Web Text)

Max Activating Token Index: 954