Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2760 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9876. Min Act: -0.1693

Data Index: 7122259 (Open Web Text)

Max Activating Token Index: 71

Text #1

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9641. Min Act: -0.1593

Data Index: 2470929 (Open Web Text)

Max Activating Token Index: 321

Text #2

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9392. Min Act: -0.1687

Data Index: 5813509 (Open Web Text)

Max Activating Token Index: 76

Text #3

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9271. Min Act: -0.1674

Data Index: 3908582 (Open Web Text)

Max Activating Token Index: 299

Text #4

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9243. Min Act: -0.1678

Data Index: 2644075 (Open Web Text)

Max Activating Token Index: 61

Text #5

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9269. Min Act: -0.1692

Data Index: 5448942 (Open Web Text)

Max Activating Token Index: 90

Text #6

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9113. Min Act: -0.1640

Data Index: 8280049 (Open Web Text)

Max Activating Token Index: 83

Text #7

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9097. Min Act: -0.1644

Data Index: 1201266 (Open Web Text)

Max Activating Token Index: 53

Text #8

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9080. Min Act: -0.1646

Data Index: 4705868 (Open Web Text)

Max Activating Token Index: 83

Text #9

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9015. Min Act: -0.1601

Data Index: 3909088 (Open Web Text)

Max Activating Token Index: 81

Text #10

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9065. Min Act: -0.1683

Data Index: 1691948 (Open Web Text)

Max Activating Token Index: 88

Text #11

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9052. Min Act: -0.1679

Data Index: 1292071 (Open Web Text)

Max Activating Token Index: 86

Text #12

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9009. Min Act: -0.1609

Data Index: 1902915 (Open Web Text)

Max Activating Token Index: 86

Text #13

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.9007. Min Act: -0.1631

Data Index: 4383268 (Open Web Text)

Max Activating Token Index: 54

Text #14

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8894. Min Act: -0.1670

Data Index: 4408521 (Open Web Text)

Max Activating Token Index: 72

Text #15

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8897. Min Act: -0.1516

Data Index: 1528841 (Open Web Text)

Max Activating Token Index: 86

Text #16

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8839. Min Act: -0.1697

Data Index: 1754542 (Open Web Text)

Max Activating Token Index: 70

Text #17

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8802. Min Act: -0.1669

Data Index: 4453898 (Open Web Text)

Max Activating Token Index: 319

Text #18

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8851. Min Act: -0.1688

Data Index: 4789811 (Open Web Text)

Max Activating Token Index: 73

Text #19

Max Range: 0.9876. Min Range: -0.9876

Max Act: 0.8844. Min Act: -0.1664

Data Index: 1355440 (Open Web Text)

Max Activating Token Index: 355