Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1622 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.5207. Min Act: -0.1700

Data Index: 7351837 (Open Web Text)

Max Activating Token Index: 292

Text #1

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.4484. Min Act: -0.1700

Data Index: 6691149 (Open Web Text)

Max Activating Token Index: 478

Text #2

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.2978. Min Act: -0.1700

Data Index: 3716210 (Open Web Text)

Max Activating Token Index: 885

Text #3

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.2305. Min Act: -0.1700

Data Index: 1614338 (Open Web Text)

Max Activating Token Index: 783

Text #4

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.2192. Min Act: -0.1700

Data Index: 1176991 (Open Web Text)

Max Activating Token Index: 251

Text #5

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1829. Min Act: -0.1700

Data Index: 266978 (Open Web Text)

Max Activating Token Index: 784

Text #6

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.2194. Min Act: -0.1700

Data Index: 8765193 (Open Web Text)

Max Activating Token Index: 155

Text #7

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.2050. Min Act: -0.1700

Data Index: 8307760 (Open Web Text)

Max Activating Token Index: 334

Text #8

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1868. Min Act: -0.1700

Data Index: 2882281 (Open Web Text)

Max Activating Token Index: 271

Text #9

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1741. Min Act: -0.1700

Data Index: 490666 (Open Web Text)

Max Activating Token Index: 219

Text #10

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1203. Min Act: -0.1700

Data Index: 5730172 (Open Web Text)

Max Activating Token Index: 284

Text #11

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0886. Min Act: -0.1700

Data Index: 2335838 (Open Web Text)

Max Activating Token Index: 420

Text #12

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0951. Min Act: -0.1700

Data Index: 4051796 (Open Web Text)

Max Activating Token Index: 317

Text #13

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1046. Min Act: -0.1700

Data Index: 7385980 (Open Web Text)

Max Activating Token Index: 691

Text #14

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1118. Min Act: -0.1700

Data Index: 8747352 (Open Web Text)

Max Activating Token Index: 68

Text #15

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.1123. Min Act: -0.1700

Data Index: 2285004 (Open Web Text)

Max Activating Token Index: 755

Text #16

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0815. Min Act: -0.1700

Data Index: 7208597 (Open Web Text)

Max Activating Token Index: 916

Text #17

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0544. Min Act: -0.1700

Data Index: 1419564 (Open Web Text)

Max Activating Token Index: 628

Text #18

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0446. Min Act: -0.1700

Data Index: 7286406 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 6.5207. Min Range: -6.5207

Max Act: 6.0340. Min Act: -0.1700

Data Index: 1767305 (Open Web Text)

Max Activating Token Index: 269