Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 166 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1967. Min Act: -0.0974

Data Index: 8093978 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.2067. Min Act: -0.0770

Data Index: 2812520 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1840. Min Act: -0.0797

Data Index: 5793499 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1797. Min Act: -0.0811

Data Index: 3698179 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1702. Min Act: -0.0818

Data Index: 550423 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1460. Min Act: -0.0727

Data Index: 1467624 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1496. Min Act: -0.0892

Data Index: 3496517 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1305. Min Act: -0.0866

Data Index: 3866646 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1330. Min Act: -0.0669

Data Index: 3759106 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1329. Min Act: -0.0763

Data Index: 385532 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1225. Min Act: -0.0816

Data Index: 369837 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1269. Min Act: -0.0814

Data Index: 3389925 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1312. Min Act: -0.1033

Data Index: 501331 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1124. Min Act: -0.0777

Data Index: 8720222 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1127. Min Act: -0.0764

Data Index: 1586810 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1162. Min Act: -0.0881

Data Index: 7636193 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1081. Min Act: -0.0965

Data Index: 2951785 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1122. Min Act: -0.0812

Data Index: 174395 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1121. Min Act: -0.0887

Data Index: 422486 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 1.2067. Min Range: -1.2067

Max Act: 1.1077. Min Act: -0.0286

Data Index: 7909192 (Open Web Text)

Max Activating Token Index: 1023