Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2127 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.7643. Min Act: -0.1700

Data Index: 7548009 (Open Web Text)

Max Activating Token Index: 816

Text #1

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.7167. Min Act: -0.1700

Data Index: 531784 (Open Web Text)

Max Activating Token Index: 795

Text #2

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6916. Min Act: -0.1700

Data Index: 8521553 (Open Web Text)

Max Activating Token Index: 929

Text #3

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6898. Min Act: -0.1700

Data Index: 218174 (Open Web Text)

Max Activating Token Index: 906

Text #4

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.7058. Min Act: -0.1700

Data Index: 4898694 (Open Web Text)

Max Activating Token Index: 945

Text #5

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6913. Min Act: -0.1700

Data Index: 5078345 (Open Web Text)

Max Activating Token Index: 936

Text #6

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6980. Min Act: -0.1700

Data Index: 6505501 (Open Web Text)

Max Activating Token Index: 905

Text #7

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6950. Min Act: -0.1700

Data Index: 6337147 (Open Web Text)

Max Activating Token Index: 410

Text #8

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6957. Min Act: -0.1700

Data Index: 2375938 (Open Web Text)

Max Activating Token Index: 964

Text #9

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6590. Min Act: -0.1700

Data Index: 495867 (Open Web Text)

Max Activating Token Index: 833

Text #10

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6471. Min Act: -0.1700

Data Index: 8438459 (Open Web Text)

Max Activating Token Index: 944

Text #11

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6652. Min Act: -0.1700

Data Index: 6087509 (Open Web Text)

Max Activating Token Index: 458

Text #12

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6791. Min Act: -0.1694

Data Index: 5230643 (Open Web Text)

Max Activating Token Index: 913

Text #13

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6763. Min Act: -0.1700

Data Index: 3643487 (Open Web Text)

Max Activating Token Index: 695

Text #14

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6825. Min Act: -0.1700

Data Index: 3192850 (Open Web Text)

Max Activating Token Index: 782

Text #15

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6795. Min Act: -0.1700

Data Index: 2687978 (Open Web Text)

Max Activating Token Index: 883

Text #16

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6858. Min Act: -0.1700

Data Index: 2512532 (Open Web Text)

Max Activating Token Index: 920

Text #17

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6701. Min Act: -0.1700

Data Index: 2030869 (Open Web Text)

Max Activating Token Index: 840

Text #18

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6361. Min Act: -0.1700

Data Index: 1811019 (Open Web Text)

Max Activating Token Index: 807

Text #19

Max Range: 4.7643. Min Range: -4.7643

Max Act: 4.6319. Min Act: -0.1700

Data Index: 602674 (Open Web Text)

Max Activating Token Index: 931