Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1049 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 8.1721. Min Range: -8.1721

Max Act: 8.1721. Min Act: -0.1700

Data Index: 8415988 (Open Web Text)

Max Activating Token Index: 355

Text #1

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.8223. Min Act: -0.1700

Data Index: 8614056 (Open Web Text)

Max Activating Token Index: 419

Text #2

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.7828. Min Act: -0.1694

Data Index: 4439213 (Open Web Text)

Max Activating Token Index: 370

Text #3

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.7984. Min Act: -0.1693

Data Index: 7076506 (Open Web Text)

Max Activating Token Index: 233

Text #4

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.7105. Min Act: -0.1700

Data Index: 2512748 (Open Web Text)

Max Activating Token Index: 698

Text #5

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.7230. Min Act: -0.1700

Data Index: 8059543 (Open Web Text)

Max Activating Token Index: 878

Text #6

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5809. Min Act: -0.1700

Data Index: 5260958 (Open Web Text)

Max Activating Token Index: 242

Text #7

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5848. Min Act: -0.1699

Data Index: 3611114 (Open Web Text)

Max Activating Token Index: 844

Text #8

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5614. Min Act: -0.1700

Data Index: 5720636 (Open Web Text)

Max Activating Token Index: 612

Text #9

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.6264. Min Act: -0.1699

Data Index: 1203703 (Open Web Text)

Max Activating Token Index: 789

Text #10

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.6201. Min Act: -0.1700

Data Index: 4447336 (Open Web Text)

Max Activating Token Index: 788

Text #11

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5576. Min Act: -0.1700

Data Index: 3327630 (Open Web Text)

Max Activating Token Index: 390

Text #12

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5721. Min Act: -0.1700

Data Index: 2034812 (Open Web Text)

Max Activating Token Index: 223

Text #13

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5802. Min Act: -0.1700

Data Index: 1671471 (Open Web Text)

Max Activating Token Index: 434

Text #14

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.4977. Min Act: -0.1700

Data Index: 5874315 (Open Web Text)

Max Activating Token Index: 296

Text #15

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.4851. Min Act: -0.1700

Data Index: 1906095 (Open Web Text)

Max Activating Token Index: 876

Text #16

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.4519. Min Act: -0.1700

Data Index: 8150108 (Open Web Text)

Max Activating Token Index: 884

Text #17

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.5047. Min Act: -0.1700

Data Index: 2446193 (Open Web Text)

Max Activating Token Index: 143

Text #18

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.4563. Min Act: -0.1700

Data Index: 2252309 (Open Web Text)

Max Activating Token Index: 523

Text #19

Max Range: 8.1721. Min Range: -8.1721

Max Act: 7.4740. Min Act: -0.1700

Data Index: 4034598 (Open Web Text)

Max Activating Token Index: 336