Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1476 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.7576. Min Act: -0.1700

Data Index: 1837220 (Open Web Text)

Max Activating Token Index: 470

Text #1

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.7532. Min Act: -0.1700

Data Index: 213006 (Open Web Text)

Max Activating Token Index: 817

Text #2

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.7289. Min Act: -0.1700

Data Index: 5844148 (Open Web Text)

Max Activating Token Index: 250

Text #3

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.7220. Min Act: -0.1700

Data Index: 2672612 (Open Web Text)

Max Activating Token Index: 398

Text #4

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6943. Min Act: -0.1700

Data Index: 8774260 (Open Web Text)

Max Activating Token Index: 365

Text #5

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6871. Min Act: -0.1700

Data Index: 5024097 (Open Web Text)

Max Activating Token Index: 549

Text #6

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6732. Min Act: -0.1700

Data Index: 6513904 (Open Web Text)

Max Activating Token Index: 827

Text #7

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6606. Min Act: -0.1700

Data Index: 7540985 (Open Web Text)

Max Activating Token Index: 464

Text #8

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6833. Min Act: -0.1700

Data Index: 8127998 (Open Web Text)

Max Activating Token Index: 561

Text #9

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6342. Min Act: -0.1700

Data Index: 1706003 (Open Web Text)

Max Activating Token Index: 269

Text #10

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6274. Min Act: -0.1700

Data Index: 3242315 (Open Web Text)

Max Activating Token Index: 883

Text #11

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6173. Min Act: -0.1700

Data Index: 4435143 (Open Web Text)

Max Activating Token Index: 326

Text #12

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6240. Min Act: -0.1700

Data Index: 8026381 (Open Web Text)

Max Activating Token Index: 660

Text #13

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6385. Min Act: -0.1700

Data Index: 5024096 (Open Web Text)

Max Activating Token Index: 546

Text #14

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6111. Min Act: -0.1700

Data Index: 1628119 (Open Web Text)

Max Activating Token Index: 18

Text #15

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6028. Min Act: -0.1700

Data Index: 3155263 (Open Web Text)

Max Activating Token Index: 163

Text #16

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6013. Min Act: -0.1700

Data Index: 6986787 (Open Web Text)

Max Activating Token Index: 666

Text #17

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.5984. Min Act: -0.1700

Data Index: 3150245 (Open Web Text)

Max Activating Token Index: 848

Text #18

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6096. Min Act: -0.1700

Data Index: 272185 (Open Web Text)

Max Activating Token Index: 156

Text #19

Max Range: 2.7576. Min Range: -2.7576

Max Act: 2.6018. Min Act: -0.1700

Data Index: 2947882 (Open Web Text)

Max Activating Token Index: 35