Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 158 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.2256. Min Act: -0.1700

Data Index: 895346 (Open Web Text)

Max Activating Token Index: 622

Text #1

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.2204. Min Act: -0.1700

Data Index: 6895669 (Open Web Text)

Max Activating Token Index: 969

Text #2

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.2054. Min Act: -0.1700

Data Index: 8404109 (Open Web Text)

Max Activating Token Index: 784

Text #3

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1579. Min Act: -0.1700

Data Index: 693208 (Open Web Text)

Max Activating Token Index: 564

Text #4

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1589. Min Act: -0.1700

Data Index: 189639 (Open Web Text)

Max Activating Token Index: 944

Text #5

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1634. Min Act: -0.1700

Data Index: 8118024 (Open Web Text)

Max Activating Token Index: 20

Text #6

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1384. Min Act: -0.1700

Data Index: 3289920 (Open Web Text)

Max Activating Token Index: 789

Text #7

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1272. Min Act: -0.1700

Data Index: 4973104 (Open Web Text)

Max Activating Token Index: 80

Text #8

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1242. Min Act: -0.1700

Data Index: 4860930 (Open Web Text)

Max Activating Token Index: 277

Text #9

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1250. Min Act: -0.1700

Data Index: 1402261 (Open Web Text)

Max Activating Token Index: 811

Text #10

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1441. Min Act: -0.1700

Data Index: 4621997 (Open Web Text)

Max Activating Token Index: 648

Text #11

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1174. Min Act: -0.1700

Data Index: 8062814 (Open Web Text)

Max Activating Token Index: 944

Text #12

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1415. Min Act: -0.1700

Data Index: 3965018 (Open Web Text)

Max Activating Token Index: 96

Text #13

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1081. Min Act: -0.1700

Data Index: 7165527 (Open Web Text)

Max Activating Token Index: 652

Text #14

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1123. Min Act: -0.1700

Data Index: 3721529 (Open Web Text)

Max Activating Token Index: 870

Text #15

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1288. Min Act: -0.1700

Data Index: 5869441 (Open Web Text)

Max Activating Token Index: 440

Text #16

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1226. Min Act: -0.1700

Data Index: 2252219 (Open Web Text)

Max Activating Token Index: 432

Text #17

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.0948. Min Act: -0.1700

Data Index: 4603839 (Open Web Text)

Max Activating Token Index: 492

Text #18

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1265. Min Act: -0.1700

Data Index: 303731 (Open Web Text)

Max Activating Token Index: 287

Text #19

Max Range: 4.2256. Min Range: -4.2256

Max Act: 4.1081. Min Act: -0.1700

Data Index: 5537232 (Open Web Text)

Max Activating Token Index: 231