Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1132 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1770. Min Act: -0.1700

Data Index: 355070 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1640. Min Act: -0.1700

Data Index: 6489638 (Open Web Text)

Max Activating Token Index: 722

Text #2

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1578. Min Act: -0.1700

Data Index: 1527072 (Open Web Text)

Max Activating Token Index: 325

Text #3

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1401. Min Act: -0.1700

Data Index: 1361190 (Open Web Text)

Max Activating Token Index: 825

Text #4

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1550. Min Act: -0.1700

Data Index: 7310715 (Open Web Text)

Max Activating Token Index: 601

Text #5

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1308. Min Act: -0.1700

Data Index: 4216151 (Open Web Text)

Max Activating Token Index: 74

Text #6

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1232. Min Act: -0.1700

Data Index: 2216254 (Open Web Text)

Max Activating Token Index: 519

Text #7

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1178. Min Act: -0.1700

Data Index: 5622101 (Open Web Text)

Max Activating Token Index: 877

Text #8

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0887. Min Act: -0.1700

Data Index: 2322574 (Open Web Text)

Max Activating Token Index: 73

Text #9

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0870. Min Act: -0.1700

Data Index: 5277675 (Open Web Text)

Max Activating Token Index: 472

Text #10

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1261. Min Act: -0.1700

Data Index: 8809498 (Open Web Text)

Max Activating Token Index: 905

Text #11

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1088. Min Act: -0.1700

Data Index: 1997695 (Open Web Text)

Max Activating Token Index: 430

Text #12

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0979. Min Act: -0.1700

Data Index: 6401055 (Open Web Text)

Max Activating Token Index: 883

Text #13

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0896. Min Act: -0.1700

Data Index: 7781733 (Open Web Text)

Max Activating Token Index: 529

Text #14

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0903. Min Act: -0.1700

Data Index: 79306 (Open Web Text)

Max Activating Token Index: 811

Text #15

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1010. Min Act: -0.1700

Data Index: 7553970 (Open Web Text)

Max Activating Token Index: 748

Text #16

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0615. Min Act: -0.1700

Data Index: 6747250 (Open Web Text)

Max Activating Token Index: 778

Text #17

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.1095. Min Act: -0.1700

Data Index: 6800708 (Open Web Text)

Max Activating Token Index: 795

Text #18

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0743. Min Act: -0.1700

Data Index: 7194546 (Open Web Text)

Max Activating Token Index: 71

Text #19

Max Range: 3.1770. Min Range: -3.1770

Max Act: 3.0657. Min Act: -0.1700

Data Index: 5294796 (Open Web Text)

Max Activating Token Index: 572