Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1559 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0346. Min Range: -5.0346

Max Act: 5.0346. Min Act: -0.1700

Data Index: 4637282 (Open Web Text)

Max Activating Token Index: 31

Text #1

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.9504. Min Act: -0.1700

Data Index: 4915873 (Open Web Text)

Max Activating Token Index: 168

Text #2

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.9031. Min Act: -0.1700

Data Index: 8423407 (Open Web Text)

Max Activating Token Index: 475

Text #3

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.9265. Min Act: -0.1700

Data Index: 4700966 (Open Web Text)

Max Activating Token Index: 840

Text #4

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.9199. Min Act: -0.1700

Data Index: 4697208 (Open Web Text)

Max Activating Token Index: 289

Text #5

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8721. Min Act: -0.1700

Data Index: 4365654 (Open Web Text)

Max Activating Token Index: 160

Text #6

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8788. Min Act: -0.1700

Data Index: 5755745 (Open Web Text)

Max Activating Token Index: 243

Text #7

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8842. Min Act: -0.1700

Data Index: 7496921 (Open Web Text)

Max Activating Token Index: 282

Text #8

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8415. Min Act: -0.1700

Data Index: 1535014 (Open Web Text)

Max Activating Token Index: 182

Text #9

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8270. Min Act: -0.1700

Data Index: 6118156 (Open Web Text)

Max Activating Token Index: 480

Text #10

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7689. Min Act: -0.1700

Data Index: 5176445 (Open Web Text)

Max Activating Token Index: 415

Text #11

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8170. Min Act: -0.1700

Data Index: 7335153 (Open Web Text)

Max Activating Token Index: 428

Text #12

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7979. Min Act: -0.1700

Data Index: 611449 (Open Web Text)

Max Activating Token Index: 890

Text #13

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7998. Min Act: -0.1700

Data Index: 7059234 (Open Web Text)

Max Activating Token Index: 1016

Text #14

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.8080. Min Act: -0.1700

Data Index: 3135780 (Open Web Text)

Max Activating Token Index: 153

Text #15

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7931. Min Act: -0.1700

Data Index: 2675970 (Open Web Text)

Max Activating Token Index: 371

Text #16

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7777. Min Act: -0.1700

Data Index: 1530981 (Open Web Text)

Max Activating Token Index: 842

Text #17

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7606. Min Act: -0.1700

Data Index: 5635967 (Open Web Text)

Max Activating Token Index: 653

Text #18

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7641. Min Act: -0.1700

Data Index: 2680790 (Open Web Text)

Max Activating Token Index: 997

Text #19

Max Range: 5.0346. Min Range: -5.0346

Max Act: 4.7539. Min Act: -0.1700

Data Index: 3641659 (Open Web Text)

Max Activating Token Index: 189