Model: GPT-2 Xl: 48 Layers, 6400 Neurons per Layer

Dataset: Open Web Text

Neuron 5200 in Layer 12

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gpt2-xl')

Text #0

Max Range: 5.3390. Min Range: -5.3390

Max Act: 5.3390. Min Act: -0.1700

Data Index: 8728963 (Open Web Text)

Max Activating Token Index: 555

Text #1

Max Range: 5.3390. Min Range: -5.3390

Max Act: 5.2453. Min Act: -0.1700

Data Index: 7635722 (Open Web Text)

Max Activating Token Index: 454

Text #2

Max Range: 5.3390. Min Range: -5.3390

Max Act: 5.0578. Min Act: -0.1700

Data Index: 7772969 (Open Web Text)

Max Activating Token Index: 458

Text #3

Max Range: 5.3390. Min Range: -5.3390

Max Act: 5.0265. Min Act: -0.1700

Data Index: 7511524 (Open Web Text)

Max Activating Token Index: 915

Text #4

Max Range: 5.3390. Min Range: -5.3390

Max Act: 5.0265. Min Act: -0.1700

Data Index: 8803283 (Open Web Text)

Max Activating Token Index: 847

Text #5

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9953. Min Act: -0.1700

Data Index: 498064 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9640. Min Act: -0.1700

Data Index: 5008002 (Open Web Text)

Max Activating Token Index: 398

Text #7

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9953. Min Act: -0.1700

Data Index: 7987987 (Open Web Text)

Max Activating Token Index: 282

Text #8

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9953. Min Act: -0.1700

Data Index: 4474654 (Open Web Text)

Max Activating Token Index: 168

Text #9

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9640. Min Act: -0.1700

Data Index: 1560761 (Open Web Text)

Max Activating Token Index: 166

Text #10

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9953. Min Act: -0.1700

Data Index: 3781734 (Open Web Text)

Max Activating Token Index: 938

Text #11

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9328. Min Act: -0.1700

Data Index: 1722041 (Open Web Text)

Max Activating Token Index: 494

Text #12

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9328. Min Act: -0.1700

Data Index: 829406 (Open Web Text)

Max Activating Token Index: 347

Text #13

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9328. Min Act: -0.1700

Data Index: 531144 (Open Web Text)

Max Activating Token Index: 355

Text #14

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9015. Min Act: -0.1700

Data Index: 529856 (Open Web Text)

Max Activating Token Index: 493

Text #15

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.8703. Min Act: -0.1700

Data Index: 4158028 (Open Web Text)

Max Activating Token Index: 875

Text #16

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9015. Min Act: -0.1700

Data Index: 622997 (Open Web Text)

Max Activating Token Index: 644

Text #17

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9015. Min Act: -0.1700

Data Index: 1992292 (Open Web Text)

Max Activating Token Index: 836

Text #18

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9015. Min Act: -0.1700

Data Index: 1380060 (Open Web Text)

Max Activating Token Index: 313

Text #19

Max Range: 5.3390. Min Range: -5.3390

Max Act: 4.9328. Min Act: -0.1699

Data Index: 7002384 (Open Web Text)

Max Activating Token Index: 725