Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 763 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.3246. Min Act: -0.1700

Data Index: 1399032 (Open Web Text)

Max Activating Token Index: 862

Text #1

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.2611. Min Act: -0.1700

Data Index: 6653966 (Open Web Text)

Max Activating Token Index: 887

Text #2

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1703. Min Act: -0.1700

Data Index: 389794 (Open Web Text)

Max Activating Token Index: 890

Text #3

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1406. Min Act: -0.1700

Data Index: 2408557 (Open Web Text)

Max Activating Token Index: 939

Text #4

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1174. Min Act: -0.1700

Data Index: 3902554 (Open Web Text)

Max Activating Token Index: 967

Text #5

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1236. Min Act: -0.1700

Data Index: 1252826 (Open Web Text)

Max Activating Token Index: 887

Text #6

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1064. Min Act: -0.1700

Data Index: 419517 (Open Web Text)

Max Activating Token Index: 709

Text #7

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.1023. Min Act: -0.1700

Data Index: 1652676 (Open Web Text)

Max Activating Token Index: 883

Text #8

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0663. Min Act: -0.1700

Data Index: 1238787 (Open Web Text)

Max Activating Token Index: 754

Text #9

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0119. Min Act: -0.1700

Data Index: 3021025 (Open Web Text)

Max Activating Token Index: 884

Text #10

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0263. Min Act: -0.1700

Data Index: 5197155 (Open Web Text)

Max Activating Token Index: 897

Text #11

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0361. Min Act: -0.1700

Data Index: 4507809 (Open Web Text)

Max Activating Token Index: 837

Text #12

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0343. Min Act: -0.1700

Data Index: 1554097 (Open Web Text)

Max Activating Token Index: 945

Text #13

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0097. Min Act: -0.1700

Data Index: 1662538 (Open Web Text)

Max Activating Token Index: 986

Text #14

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0007. Min Act: -0.1700

Data Index: 4328589 (Open Web Text)

Max Activating Token Index: 912

Text #15

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0044. Min Act: -0.1700

Data Index: 1452641 (Open Web Text)

Max Activating Token Index: 925

Text #16

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0146. Min Act: -0.1700

Data Index: 3552301 (Open Web Text)

Max Activating Token Index: 967

Text #17

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0288. Min Act: -0.1700

Data Index: 2157678 (Open Web Text)

Max Activating Token Index: 935

Text #18

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0026. Min Act: -0.1700

Data Index: 3891441 (Open Web Text)

Max Activating Token Index: 957

Text #19

Max Range: 3.3246. Min Range: -3.3246

Max Act: 3.0115. Min Act: -0.1700

Data Index: 2388698 (Open Web Text)

Max Activating Token Index: 929