Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 534 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.7439. Min Act: -0.1700

Data Index: 390024 (Open Web Text)

Max Activating Token Index: 924

Text #1

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.7399. Min Act: -0.1700

Data Index: 6536637 (Open Web Text)

Max Activating Token Index: 1001

Text #2

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.6111. Min Act: -0.1700

Data Index: 2967275 (Open Web Text)

Max Activating Token Index: 642

Text #3

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.6148. Min Act: -0.1700

Data Index: 1329131 (Open Web Text)

Max Activating Token Index: 725

Text #4

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.6130. Min Act: -0.1700

Data Index: 8028590 (Open Web Text)

Max Activating Token Index: 329

Text #5

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.6153. Min Act: -0.1700

Data Index: 1124300 (Open Web Text)

Max Activating Token Index: 272

Text #6

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5998. Min Act: -0.1700

Data Index: 3772887 (Open Web Text)

Max Activating Token Index: 335

Text #7

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.6127. Min Act: -0.1700

Data Index: 8556347 (Open Web Text)

Max Activating Token Index: 427

Text #8

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5863. Min Act: -0.1700

Data Index: 6299131 (Open Web Text)

Max Activating Token Index: 727

Text #9

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5973. Min Act: -0.1700

Data Index: 4581458 (Open Web Text)

Max Activating Token Index: 135

Text #10

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5906. Min Act: -0.1700

Data Index: 4603335 (Open Web Text)

Max Activating Token Index: 971

Text #11

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5895. Min Act: -0.1700

Data Index: 7108461 (Open Web Text)

Max Activating Token Index: 240

Text #12

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5471. Min Act: -0.1700

Data Index: 5488745 (Open Web Text)

Max Activating Token Index: 463

Text #13

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5517. Min Act: -0.1700

Data Index: 6398628 (Open Web Text)

Max Activating Token Index: 430

Text #14

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5566. Min Act: -0.1700

Data Index: 4817366 (Open Web Text)

Max Activating Token Index: 529

Text #15

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5602. Min Act: -0.1700

Data Index: 7737693 (Open Web Text)

Max Activating Token Index: 293

Text #16

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5260. Min Act: -0.1700

Data Index: 2896768 (Open Web Text)

Max Activating Token Index: 106

Text #17

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5299. Min Act: -0.1700

Data Index: 1373038 (Open Web Text)

Max Activating Token Index: 617

Text #18

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5282. Min Act: -0.1700

Data Index: 4269612 (Open Web Text)

Max Activating Token Index: 772

Text #19

Max Range: 3.7439. Min Range: -3.7439

Max Act: 3.5051. Min Act: -0.1700

Data Index: 6923197 (Open Web Text)

Max Activating Token Index: 502