Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2814 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.7173. Min Act: -0.1700

Data Index: 1016106 (Open Web Text)

Max Activating Token Index: 485

Text #1

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.7120. Min Act: -0.1700

Data Index: 2908400 (Open Web Text)

Max Activating Token Index: 515

Text #2

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.7026. Min Act: -0.1700

Data Index: 1231507 (Open Web Text)

Max Activating Token Index: 962

Text #3

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.6185. Min Act: -0.1700

Data Index: 1206727 (Open Web Text)

Max Activating Token Index: 745

Text #4

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.6506. Min Act: -0.1700

Data Index: 3746947 (Open Web Text)

Max Activating Token Index: 997

Text #5

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.6523. Min Act: -0.1700

Data Index: 7528014 (Open Web Text)

Max Activating Token Index: 989

Text #6

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.6329. Min Act: -0.1700

Data Index: 1439916 (Open Web Text)

Max Activating Token Index: 667

Text #7

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5684. Min Act: -0.1700

Data Index: 6829761 (Open Web Text)

Max Activating Token Index: 973

Text #8

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5626. Min Act: -0.1700

Data Index: 2615224 (Open Web Text)

Max Activating Token Index: 806

Text #9

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5994. Min Act: -0.1700

Data Index: 4426809 (Open Web Text)

Max Activating Token Index: 829

Text #10

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5748. Min Act: -0.1700

Data Index: 8633233 (Open Web Text)

Max Activating Token Index: 721

Text #11

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5624. Min Act: -0.1700

Data Index: 2357434 (Open Web Text)

Max Activating Token Index: 785

Text #12

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5826. Min Act: -0.1700

Data Index: 6143036 (Open Web Text)

Max Activating Token Index: 549

Text #13

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5635. Min Act: -0.1700

Data Index: 3326806 (Open Web Text)

Max Activating Token Index: 412

Text #14

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5358. Min Act: -0.1700

Data Index: 3417682 (Open Web Text)

Max Activating Token Index: 566

Text #15

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5381. Min Act: -0.1700

Data Index: 2375557 (Open Web Text)

Max Activating Token Index: 176

Text #16

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5586. Min Act: -0.1700

Data Index: 131815 (Open Web Text)

Max Activating Token Index: 781

Text #17

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5272. Min Act: -0.1700

Data Index: 989080 (Open Web Text)

Max Activating Token Index: 765

Text #18

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5293. Min Act: -0.1700

Data Index: 2296235 (Open Web Text)

Max Activating Token Index: 725

Text #19

Max Range: 3.7173. Min Range: -3.7173

Max Act: 3.5729. Min Act: -0.1700

Data Index: 4728200 (Open Web Text)

Max Activating Token Index: 1005