Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1622 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5778. Min Act: -0.1700

Data Index: 2526886 (Open Web Text)

Max Activating Token Index: 515

Text #1

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5682. Min Act: -0.1700

Data Index: 7674403 (Open Web Text)

Max Activating Token Index: 536

Text #2

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5693. Min Act: -0.1700

Data Index: 7835552 (Open Web Text)

Max Activating Token Index: 375

Text #3

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5682. Min Act: -0.1699

Data Index: 7891155 (Open Web Text)

Max Activating Token Index: 106

Text #4

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5558. Min Act: -0.1700

Data Index: 7835554 (Open Web Text)

Max Activating Token Index: 196

Text #5

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5537. Min Act: -0.1700

Data Index: 2892646 (Open Web Text)

Max Activating Token Index: 390

Text #6

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5511. Min Act: -0.1700

Data Index: 5146225 (Open Web Text)

Max Activating Token Index: 96

Text #7

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5542. Min Act: -0.1700

Data Index: 7333186 (Open Web Text)

Max Activating Token Index: 335

Text #8

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5415. Min Act: -0.1700

Data Index: 3219769 (Open Web Text)

Max Activating Token Index: 178

Text #9

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5341. Min Act: -0.1700

Data Index: 5748753 (Open Web Text)

Max Activating Token Index: 450

Text #10

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5438. Min Act: -0.1700

Data Index: 646621 (Open Web Text)

Max Activating Token Index: 817

Text #11

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5203. Min Act: -0.1700

Data Index: 1256849 (Open Web Text)

Max Activating Token Index: 125

Text #12

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5319. Min Act: -0.1700

Data Index: 2097314 (Open Web Text)

Max Activating Token Index: 911

Text #13

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5211. Min Act: -0.1700

Data Index: 907442 (Open Web Text)

Max Activating Token Index: 280

Text #14

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5222. Min Act: -0.1700

Data Index: 6054925 (Open Web Text)

Max Activating Token Index: 195

Text #15

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5077. Min Act: -0.1700

Data Index: 2440850 (Open Web Text)

Max Activating Token Index: 180

Text #16

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5041. Min Act: -0.1700

Data Index: 6688420 (Open Web Text)

Max Activating Token Index: 672

Text #17

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5060. Min Act: -0.1698

Data Index: 4602270 (Open Web Text)

Max Activating Token Index: 389

Text #18

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5129. Min Act: -0.1700

Data Index: 7052654 (Open Web Text)

Max Activating Token Index: 724

Text #19

Max Range: 1.5778. Min Range: -1.5778

Max Act: 1.5036. Min Act: -0.1700

Data Index: 3881593 (Open Web Text)

Max Activating Token Index: 662