Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2040 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9704. Min Act: -0.1700

Data Index: 3696011 (Open Web Text)

Max Activating Token Index: 571

Text #1

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9144. Min Act: -0.1700

Data Index: 8760434 (Open Web Text)

Max Activating Token Index: 356

Text #2

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9760. Min Act: -0.1700

Data Index: 4750147 (Open Web Text)

Max Activating Token Index: 175

Text #3

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9178. Min Act: -0.1700

Data Index: 8316496 (Open Web Text)

Max Activating Token Index: 235

Text #4

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8979. Min Act: -0.1700

Data Index: 8191252 (Open Web Text)

Max Activating Token Index: 64

Text #5

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8935. Min Act: -0.1700

Data Index: 2789476 (Open Web Text)

Max Activating Token Index: 325

Text #6

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8955. Min Act: -0.1700

Data Index: 4639116 (Open Web Text)

Max Activating Token Index: 487

Text #7

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9289. Min Act: -0.1700

Data Index: 963535 (Open Web Text)

Max Activating Token Index: 458

Text #8

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8740. Min Act: -0.1700

Data Index: 4618967 (Open Web Text)

Max Activating Token Index: 850

Text #9

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9138. Min Act: -0.1700

Data Index: 4412167 (Open Web Text)

Max Activating Token Index: 198

Text #10

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9192. Min Act: -0.1700

Data Index: 7582194 (Open Web Text)

Max Activating Token Index: 831

Text #11

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9145. Min Act: -0.1700

Data Index: 7442678 (Open Web Text)

Max Activating Token Index: 479

Text #12

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9002. Min Act: -0.1700

Data Index: 541401 (Open Web Text)

Max Activating Token Index: 165

Text #13

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9068. Min Act: -0.1700

Data Index: 7218840 (Open Web Text)

Max Activating Token Index: 135

Text #14

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8124. Min Act: -0.1700

Data Index: 6989642 (Open Web Text)

Max Activating Token Index: 607

Text #15

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9206. Min Act: -0.1700

Data Index: 5540481 (Open Web Text)

Max Activating Token Index: 72

Text #16

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.9069. Min Act: -0.1700

Data Index: 3931373 (Open Web Text)

Max Activating Token Index: 418

Text #17

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8671. Min Act: -0.1700

Data Index: 182697 (Open Web Text)

Max Activating Token Index: 154

Text #18

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8684. Min Act: -0.1700

Data Index: 2703042 (Open Web Text)

Max Activating Token Index: 729

Text #19

Max Range: 3.9760. Min Range: -3.9760

Max Act: 3.8325. Min Act: -0.1700

Data Index: 2357158 (Open Web Text)

Max Activating Token Index: 519