Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 728 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.7553. Min Act: -0.1700

Data Index: 3862357 (Open Web Text)

Max Activating Token Index: 210

Text #1

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6836. Min Act: -0.1700

Data Index: 7972314 (Open Web Text)

Max Activating Token Index: 585

Text #2

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6987. Min Act: -0.1700

Data Index: 5640841 (Open Web Text)

Max Activating Token Index: 125

Text #3

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6583. Min Act: -0.1700

Data Index: 8522488 (Open Web Text)

Max Activating Token Index: 635

Text #4

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6550. Min Act: -0.1700

Data Index: 5732148 (Open Web Text)

Max Activating Token Index: 756

Text #5

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6437. Min Act: -0.1700

Data Index: 8168390 (Open Web Text)

Max Activating Token Index: 383

Text #6

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6520. Min Act: -0.1700

Data Index: 8521876 (Open Web Text)

Max Activating Token Index: 952

Text #7

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6495. Min Act: -0.1700

Data Index: 6141718 (Open Web Text)

Max Activating Token Index: 309

Text #8

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6300. Min Act: -0.1700

Data Index: 4926445 (Open Web Text)

Max Activating Token Index: 701

Text #9

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6398. Min Act: -0.1700

Data Index: 7242976 (Open Web Text)

Max Activating Token Index: 676

Text #10

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6400. Min Act: -0.1700

Data Index: 8334905 (Open Web Text)

Max Activating Token Index: 371

Text #11

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6396. Min Act: -0.1700

Data Index: 3457045 (Open Web Text)

Max Activating Token Index: 273

Text #12

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6269. Min Act: -0.1700

Data Index: 6573289 (Open Web Text)

Max Activating Token Index: 277

Text #13

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6101. Min Act: -0.1700

Data Index: 2884007 (Open Web Text)

Max Activating Token Index: 861

Text #14

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6188. Min Act: -0.1700

Data Index: 403475 (Open Web Text)

Max Activating Token Index: 503

Text #15

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6085. Min Act: -0.1700

Data Index: 7109798 (Open Web Text)

Max Activating Token Index: 670

Text #16

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.6280. Min Act: -0.1700

Data Index: 3686897 (Open Web Text)

Max Activating Token Index: 667

Text #17

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.5986. Min Act: -0.1700

Data Index: 5011981 (Open Web Text)

Max Activating Token Index: 965

Text #18

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.5995. Min Act: -0.1700

Data Index: 6595743 (Open Web Text)

Max Activating Token Index: 403

Text #19

Max Range: 3.7553. Min Range: -3.7553

Max Act: 3.5878. Min Act: -0.1700

Data Index: 6697839 (Open Web Text)

Max Activating Token Index: 566