Model: GPT-2 Xl: 48 Layers, 6400 Neurons per Layer

Dataset: Open Web Text

Neuron 4471 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gpt2-xl')

Text #0

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.8320. Min Act: -0.1700

Data Index: 753167 (Open Web Text)

Max Activating Token Index: 349

Text #1

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.8320. Min Act: -0.1700

Data Index: 1453686 (Open Web Text)

Max Activating Token Index: 70

Text #2

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.7070. Min Act: -0.1700

Data Index: 1191190 (Open Web Text)

Max Activating Token Index: 456

Text #3

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.5820. Min Act: -0.1700

Data Index: 1446209 (Open Web Text)

Max Activating Token Index: 339

Text #4

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.5508. Min Act: -0.1700

Data Index: 6229700 (Open Web Text)

Max Activating Token Index: 733

Text #5

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.4570. Min Act: -0.1700

Data Index: 7684636 (Open Web Text)

Max Activating Token Index: 72

Text #6

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.4258. Min Act: -0.1700

Data Index: 2256020 (Open Web Text)

Max Activating Token Index: 24

Text #7

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3633. Min Act: -0.1700

Data Index: 6667515 (Open Web Text)

Max Activating Token Index: 363

Text #8

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3008. Min Act: -0.1700

Data Index: 4355071 (Open Web Text)

Max Activating Token Index: 638

Text #9

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3320. Min Act: -0.1700

Data Index: 4501629 (Open Web Text)

Max Activating Token Index: 71

Text #10

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3320. Min Act: -0.1700

Data Index: 899047 (Open Web Text)

Max Activating Token Index: 302

Text #11

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3320. Min Act: -0.1700

Data Index: 1442583 (Open Web Text)

Max Activating Token Index: 835

Text #12

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3320. Min Act: -0.1700

Data Index: 3530079 (Open Web Text)

Max Activating Token Index: 181

Text #13

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.3008. Min Act: -0.1700

Data Index: 5352248 (Open Web Text)

Max Activating Token Index: 772

Text #14

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2383. Min Act: -0.1700

Data Index: 3781123 (Open Web Text)

Max Activating Token Index: 718

Text #15

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2383. Min Act: -0.1700

Data Index: 1000409 (Open Web Text)

Max Activating Token Index: 705

Text #16

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2383. Min Act: -0.1700

Data Index: 5020048 (Open Web Text)

Max Activating Token Index: 942

Text #17

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2070. Min Act: -0.1700

Data Index: 8306320 (Open Web Text)

Max Activating Token Index: 716

Text #18

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2070. Min Act: -0.1700

Data Index: 7303770 (Open Web Text)

Max Activating Token Index: 267

Text #19

Max Range: 5.8320. Min Range: -5.8320

Max Act: 5.2070. Min Act: -0.1700

Data Index: 7285480 (Open Web Text)

Max Activating Token Index: 484