Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 483 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.7429. Min Act: -0.1700

Data Index: 1444984 (Open Web Text)

Max Activating Token Index: 472

Text #1

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.7195. Min Act: -0.1700

Data Index: 7302242 (Open Web Text)

Max Activating Token Index: 1020

Text #2

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.7569. Min Act: -0.1700

Data Index: 678301 (Open Web Text)

Max Activating Token Index: 1000

Text #3

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.7130. Min Act: -0.1700

Data Index: 2358474 (Open Web Text)

Max Activating Token Index: 65

Text #4

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.7038. Min Act: -0.1700

Data Index: 917581 (Open Web Text)

Max Activating Token Index: 87

Text #5

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.6600. Min Act: -0.1700

Data Index: 1456895 (Open Web Text)

Max Activating Token Index: 886

Text #6

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.6239. Min Act: -0.1700

Data Index: 5457802 (Open Web Text)

Max Activating Token Index: 927

Text #7

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5992. Min Act: -0.1700

Data Index: 6114698 (Open Web Text)

Max Activating Token Index: 490

Text #8

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.6144. Min Act: -0.1700

Data Index: 8534865 (Open Web Text)

Max Activating Token Index: 610

Text #9

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5924. Min Act: -0.1700

Data Index: 1617068 (Open Web Text)

Max Activating Token Index: 320

Text #10

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5705. Min Act: -0.1700

Data Index: 2289400 (Open Web Text)

Max Activating Token Index: 889

Text #11

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5598. Min Act: -0.1700

Data Index: 6817132 (Open Web Text)

Max Activating Token Index: 116

Text #12

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5739. Min Act: -0.1700

Data Index: 6951953 (Open Web Text)

Max Activating Token Index: 1017

Text #13

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5598. Min Act: -0.1700

Data Index: 6608555 (Open Web Text)

Max Activating Token Index: 1018

Text #14

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5294. Min Act: -0.1700

Data Index: 5896706 (Open Web Text)

Max Activating Token Index: 794

Text #15

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5480. Min Act: -0.1700

Data Index: 1362681 (Open Web Text)

Max Activating Token Index: 501

Text #16

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5242. Min Act: -0.1700

Data Index: 3573139 (Open Web Text)

Max Activating Token Index: 92

Text #17

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5403. Min Act: -0.1700

Data Index: 6006777 (Open Web Text)

Max Activating Token Index: 714

Text #18

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5124. Min Act: -0.1700

Data Index: 7827430 (Open Web Text)

Max Activating Token Index: 928

Text #19

Max Range: 4.7569. Min Range: -4.7569

Max Act: 4.5552. Min Act: -0.1700

Data Index: 3747720 (Open Web Text)

Max Activating Token Index: 265