Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1693 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1932. Min Range: -6.1932

Max Act: 6.1932. Min Act: -0.1700

Data Index: 5556094 (Open Web Text)

Max Activating Token Index: 433

Text #1

Max Range: 6.1932. Min Range: -6.1932

Max Act: 6.0374. Min Act: -0.1700

Data Index: 8432913 (Open Web Text)

Max Activating Token Index: 329

Text #2

Max Range: 6.1932. Min Range: -6.1932

Max Act: 6.0546. Min Act: -0.1700

Data Index: 1085652 (Open Web Text)

Max Activating Token Index: 668

Text #3

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9706. Min Act: -0.1700

Data Index: 4931968 (Open Web Text)

Max Activating Token Index: 405

Text #4

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9175. Min Act: -0.1700

Data Index: 220698 (Open Web Text)

Max Activating Token Index: 872

Text #5

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.8837. Min Act: -0.1700

Data Index: 6283792 (Open Web Text)

Max Activating Token Index: 291

Text #6

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9440. Min Act: -0.1700

Data Index: 6299450 (Open Web Text)

Max Activating Token Index: 966

Text #7

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9459. Min Act: -0.1700

Data Index: 6031942 (Open Web Text)

Max Activating Token Index: 690

Text #8

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9057. Min Act: -0.1700

Data Index: 5088885 (Open Web Text)

Max Activating Token Index: 85

Text #9

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9382. Min Act: -0.1700

Data Index: 6663334 (Open Web Text)

Max Activating Token Index: 597

Text #10

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9085. Min Act: -0.1700

Data Index: 6336881 (Open Web Text)

Max Activating Token Index: 209

Text #11

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9158. Min Act: -0.1700

Data Index: 463839 (Open Web Text)

Max Activating Token Index: 185

Text #12

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.8845. Min Act: -0.1700

Data Index: 6449036 (Open Web Text)

Max Activating Token Index: 1017

Text #13

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.9079. Min Act: -0.1700

Data Index: 4992987 (Open Web Text)

Max Activating Token Index: 490

Text #14

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.8488. Min Act: -0.1700

Data Index: 7290241 (Open Web Text)

Max Activating Token Index: 665

Text #15

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.8129. Min Act: -0.1700

Data Index: 5398730 (Open Web Text)

Max Activating Token Index: 344

Text #16

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.8248. Min Act: -0.1700

Data Index: 227915 (Open Web Text)

Max Activating Token Index: 482

Text #17

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.7994. Min Act: -0.1700

Data Index: 5633390 (Open Web Text)

Max Activating Token Index: 862

Text #18

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.7719. Min Act: -0.1700

Data Index: 3639531 (Open Web Text)

Max Activating Token Index: 1006

Text #19

Max Range: 6.1932. Min Range: -6.1932

Max Act: 5.7614. Min Act: -0.1700

Data Index: 7947311 (Open Web Text)

Max Activating Token Index: 971