Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 622 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.5046. Min Act: -0.1700

Data Index: 968314 (Open Web Text)

Max Activating Token Index: 568

Text #1

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.5198. Min Act: -0.1700

Data Index: 7386331 (Open Web Text)

Max Activating Token Index: 41

Text #2

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4196. Min Act: -0.1700

Data Index: 292209 (Open Web Text)

Max Activating Token Index: 595

Text #3

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4474. Min Act: -0.1700

Data Index: 6250130 (Open Web Text)

Max Activating Token Index: 477

Text #4

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4304. Min Act: -0.1700

Data Index: 6559952 (Open Web Text)

Max Activating Token Index: 399

Text #5

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3819. Min Act: -0.1700

Data Index: 7295788 (Open Web Text)

Max Activating Token Index: 591

Text #6

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4126. Min Act: -0.1700

Data Index: 6521739 (Open Web Text)

Max Activating Token Index: 26

Text #7

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4116. Min Act: -0.1700

Data Index: 3187848 (Open Web Text)

Max Activating Token Index: 390

Text #8

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4184. Min Act: -0.1700

Data Index: 5595149 (Open Web Text)

Max Activating Token Index: 132

Text #9

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4215. Min Act: -0.1700

Data Index: 4270965 (Open Web Text)

Max Activating Token Index: 25

Text #10

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.4291. Min Act: -0.1700

Data Index: 8479719 (Open Web Text)

Max Activating Token Index: 609

Text #11

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3776. Min Act: -0.1700

Data Index: 5428948 (Open Web Text)

Max Activating Token Index: 440

Text #12

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3726. Min Act: -0.1700

Data Index: 5287463 (Open Web Text)

Max Activating Token Index: 279

Text #13

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3842. Min Act: -0.1700

Data Index: 2703325 (Open Web Text)

Max Activating Token Index: 892

Text #14

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3855. Min Act: -0.1700

Data Index: 5771278 (Open Web Text)

Max Activating Token Index: 16

Text #15

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3795. Min Act: -0.1700

Data Index: 4434978 (Open Web Text)

Max Activating Token Index: 599

Text #16

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3832. Min Act: -0.1700

Data Index: 1727756 (Open Web Text)

Max Activating Token Index: 22

Text #17

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3675. Min Act: -0.1700

Data Index: 6257640 (Open Web Text)

Max Activating Token Index: 127

Text #18

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3953. Min Act: -0.1700

Data Index: 4338797 (Open Web Text)

Max Activating Token Index: 538

Text #19

Max Range: 4.5198. Min Range: -4.5198

Max Act: 4.3818. Min Act: -0.1700

Data Index: 2009240 (Open Web Text)

Max Activating Token Index: 877