Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 175 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.9885. Min Act: -0.1700

Data Index: 8492607 (Open Web Text)

Max Activating Token Index: 832

Text #1

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.7615. Min Act: -0.1700

Data Index: 3049102 (Open Web Text)

Max Activating Token Index: 621

Text #2

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.7563. Min Act: -0.1700

Data Index: 5490417 (Open Web Text)

Max Activating Token Index: 173

Text #3

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.6783. Min Act: -0.1700

Data Index: 5162182 (Open Web Text)

Max Activating Token Index: 371

Text #4

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.6955. Min Act: -0.1700

Data Index: 7993818 (Open Web Text)

Max Activating Token Index: 1014

Text #5

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.6658. Min Act: -0.1700

Data Index: 755191 (Open Web Text)

Max Activating Token Index: 868

Text #6

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.7027. Min Act: -0.1700

Data Index: 3808519 (Open Web Text)

Max Activating Token Index: 571

Text #7

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.7033. Min Act: -0.1700

Data Index: 2871394 (Open Web Text)

Max Activating Token Index: 802

Text #8

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.6205. Min Act: -0.1700

Data Index: 3652136 (Open Web Text)

Max Activating Token Index: 362

Text #9

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5953. Min Act: -0.1700

Data Index: 2926418 (Open Web Text)

Max Activating Token Index: 1008

Text #10

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5594. Min Act: -0.1700

Data Index: 1089336 (Open Web Text)

Max Activating Token Index: 318

Text #11

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5663. Min Act: -0.1700

Data Index: 5231805 (Open Web Text)

Max Activating Token Index: 665

Text #12

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5104. Min Act: -0.1700

Data Index: 8687308 (Open Web Text)

Max Activating Token Index: 555

Text #13

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5404. Min Act: -0.1700

Data Index: 3843378 (Open Web Text)

Max Activating Token Index: 519

Text #14

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5152. Min Act: -0.1700

Data Index: 5368046 (Open Web Text)

Max Activating Token Index: 341

Text #15

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5053. Min Act: -0.1700

Data Index: 5020894 (Open Web Text)

Max Activating Token Index: 580

Text #16

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5147. Min Act: -0.1700

Data Index: 8168894 (Open Web Text)

Max Activating Token Index: 732

Text #17

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5457. Min Act: -0.1700

Data Index: 5804932 (Open Web Text)

Max Activating Token Index: 1016

Text #18

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.5041. Min Act: -0.1700

Data Index: 5614610 (Open Web Text)

Max Activating Token Index: 339

Text #19

Max Range: 4.9885. Min Range: -4.9885

Max Act: 4.4798. Min Act: -0.1700

Data Index: 4072169 (Open Web Text)

Max Activating Token Index: 807