Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2804 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.5139. Min Act: -0.1700

Data Index: 2072972 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4824. Min Act: -0.1700

Data Index: 4852404 (Open Web Text)

Max Activating Token Index: 345

Text #2

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4916. Min Act: -0.1700

Data Index: 2802796 (Open Web Text)

Max Activating Token Index: 749

Text #3

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4455. Min Act: -0.1700

Data Index: 7664654 (Open Web Text)

Max Activating Token Index: 182

Text #4

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4623. Min Act: -0.1700

Data Index: 8278159 (Open Web Text)

Max Activating Token Index: 797

Text #5

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4615. Min Act: -0.1700

Data Index: 210594 (Open Web Text)

Max Activating Token Index: 138

Text #6

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4315. Min Act: -0.1700

Data Index: 3587228 (Open Web Text)

Max Activating Token Index: 537

Text #7

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.4225. Min Act: -0.1700

Data Index: 3829359 (Open Web Text)

Max Activating Token Index: 14

Text #8

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3868. Min Act: -0.1700

Data Index: 8398900 (Open Web Text)

Max Activating Token Index: 423

Text #9

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3956. Min Act: -0.1700

Data Index: 6981060 (Open Web Text)

Max Activating Token Index: 499

Text #10

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3991. Min Act: -0.1700

Data Index: 3222227 (Open Web Text)

Max Activating Token Index: 849

Text #11

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3872. Min Act: -0.1700

Data Index: 283239 (Open Web Text)

Max Activating Token Index: 431

Text #12

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3696. Min Act: -0.1700

Data Index: 3740966 (Open Web Text)

Max Activating Token Index: 32

Text #13

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3606. Min Act: -0.1700

Data Index: 8671713 (Open Web Text)

Max Activating Token Index: 786

Text #14

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3440. Min Act: -0.1700

Data Index: 6586620 (Open Web Text)

Max Activating Token Index: 989

Text #15

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3466. Min Act: -0.1700

Data Index: 5187701 (Open Web Text)

Max Activating Token Index: 125

Text #16

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3646. Min Act: -0.1700

Data Index: 6417598 (Open Web Text)

Max Activating Token Index: 819

Text #17

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3685. Min Act: -0.1700

Data Index: 2802795 (Open Web Text)

Max Activating Token Index: 1006

Text #18

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3241. Min Act: -0.1700

Data Index: 427273 (Open Web Text)

Max Activating Token Index: 920

Text #19

Max Range: 3.5139. Min Range: -3.5139

Max Act: 3.3270. Min Act: -0.1700

Data Index: 5095734 (Open Web Text)

Max Activating Token Index: 419