Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2101 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.8155. Min Act: -0.1700

Data Index: 4391159 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6318. Min Act: -0.1700

Data Index: 6474877 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6226. Min Act: -0.1700

Data Index: 6194724 (Open Web Text)

Max Activating Token Index: 210

Text #3

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6384. Min Act: -0.1700

Data Index: 1433834 (Open Web Text)

Max Activating Token Index: 67

Text #4

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5747. Min Act: -0.1700

Data Index: 3426286 (Open Web Text)

Max Activating Token Index: 933

Text #5

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6032. Min Act: -0.1700

Data Index: 4652865 (Open Web Text)

Max Activating Token Index: 709

Text #6

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5861. Min Act: -0.1700

Data Index: 152509 (Open Web Text)

Max Activating Token Index: 637

Text #7

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5892. Min Act: -0.1700

Data Index: 8303864 (Open Web Text)

Max Activating Token Index: 819

Text #8

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6010. Min Act: -0.1700

Data Index: 6239389 (Open Web Text)

Max Activating Token Index: 465

Text #9

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.6079. Min Act: -0.1700

Data Index: 1261913 (Open Web Text)

Max Activating Token Index: 647

Text #10

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5038. Min Act: -0.1700

Data Index: 2091476 (Open Web Text)

Max Activating Token Index: 148

Text #11

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5841. Min Act: -0.1700

Data Index: 5950962 (Open Web Text)

Max Activating Token Index: 496

Text #12

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5626. Min Act: -0.1700

Data Index: 3999339 (Open Web Text)

Max Activating Token Index: 340

Text #13

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5495. Min Act: -0.1700

Data Index: 6617601 (Open Web Text)

Max Activating Token Index: 359

Text #14

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5417. Min Act: -0.1700

Data Index: 2584557 (Open Web Text)

Max Activating Token Index: 918

Text #15

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.4504. Min Act: -0.1700

Data Index: 3727301 (Open Web Text)

Max Activating Token Index: 997

Text #16

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.4545. Min Act: -0.1700

Data Index: 7901438 (Open Web Text)

Max Activating Token Index: 694

Text #17

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5140. Min Act: -0.1700

Data Index: 2565160 (Open Web Text)

Max Activating Token Index: 603

Text #18

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5030. Min Act: -0.1700

Data Index: 4105561 (Open Web Text)

Max Activating Token Index: 626

Text #19

Max Range: 2.8155. Min Range: -2.8155

Max Act: 2.5104. Min Act: -0.1700

Data Index: 1953864 (Open Web Text)

Max Activating Token Index: 334