Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 221 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1813. Min Act: -0.1700

Data Index: 828547 (Open Web Text)

Max Activating Token Index: 748

Text #1

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1466. Min Act: -0.1700

Data Index: 1856693 (Open Web Text)

Max Activating Token Index: 384

Text #2

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1463. Min Act: -0.1700

Data Index: 1166210 (Open Web Text)

Max Activating Token Index: 192

Text #3

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1329. Min Act: -0.1700

Data Index: 7628509 (Open Web Text)

Max Activating Token Index: 750

Text #4

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1531. Min Act: -0.1700

Data Index: 917465 (Open Web Text)

Max Activating Token Index: 125

Text #5

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1670. Min Act: -0.1700

Data Index: 6854382 (Open Web Text)

Max Activating Token Index: 639

Text #6

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1420. Min Act: -0.1700

Data Index: 760243 (Open Web Text)

Max Activating Token Index: 500

Text #7

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1376. Min Act: -0.1700

Data Index: 6283478 (Open Web Text)

Max Activating Token Index: 874

Text #8

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1716. Min Act: -0.1700

Data Index: 4142768 (Open Web Text)

Max Activating Token Index: 116

Text #9

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1535. Min Act: -0.1700

Data Index: 1867962 (Open Web Text)

Max Activating Token Index: 518

Text #10

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1341. Min Act: -0.1700

Data Index: 102917 (Open Web Text)

Max Activating Token Index: 857

Text #11

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1303. Min Act: -0.1700

Data Index: 1241950 (Open Web Text)

Max Activating Token Index: 610

Text #12

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1201. Min Act: -0.1700

Data Index: 1286831 (Open Web Text)

Max Activating Token Index: 176

Text #13

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1123. Min Act: -0.1700

Data Index: 1426390 (Open Web Text)

Max Activating Token Index: 181

Text #14

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1119. Min Act: -0.1700

Data Index: 1453347 (Open Web Text)

Max Activating Token Index: 1011

Text #15

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1187. Min Act: -0.1700

Data Index: 1622149 (Open Web Text)

Max Activating Token Index: 832

Text #16

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1204. Min Act: -0.1700

Data Index: 279054 (Open Web Text)

Max Activating Token Index: 472

Text #17

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1276. Min Act: -0.1700

Data Index: 1690801 (Open Web Text)

Max Activating Token Index: 173

Text #18

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1277. Min Act: -0.1700

Data Index: 320706 (Open Web Text)

Max Activating Token Index: 779

Text #19

Max Range: 4.1813. Min Range: -4.1813

Max Act: 4.1235. Min Act: -0.1700

Data Index: 25938 (Open Web Text)

Max Activating Token Index: 520