Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3067 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5454. Min Act: -0.1700

Data Index: 3003087 (Open Web Text)

Max Activating Token Index: 473

Text #1

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5555. Min Act: -0.1700

Data Index: 2172549 (Open Web Text)

Max Activating Token Index: 826

Text #2

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5335. Min Act: -0.1700

Data Index: 7538625 (Open Web Text)

Max Activating Token Index: 502

Text #3

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5422. Min Act: -0.1700

Data Index: 3849027 (Open Web Text)

Max Activating Token Index: 940

Text #4

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5292. Min Act: -0.1700

Data Index: 348290 (Open Web Text)

Max Activating Token Index: 603

Text #5

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5313. Min Act: -0.1700

Data Index: 3003086 (Open Web Text)

Max Activating Token Index: 446

Text #6

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5318. Min Act: -0.1700

Data Index: 4747472 (Open Web Text)

Max Activating Token Index: 916

Text #7

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5353. Min Act: -0.1700

Data Index: 8530714 (Open Web Text)

Max Activating Token Index: 918

Text #8

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5196. Min Act: -0.1700

Data Index: 7986515 (Open Web Text)

Max Activating Token Index: 247

Text #9

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5246. Min Act: -0.1700

Data Index: 7078493 (Open Web Text)

Max Activating Token Index: 672

Text #10

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5373. Min Act: -0.1700

Data Index: 6282165 (Open Web Text)

Max Activating Token Index: 431

Text #11

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5248. Min Act: -0.1700

Data Index: 6282166 (Open Web Text)

Max Activating Token Index: 456

Text #12

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5114. Min Act: -0.1700

Data Index: 7986516 (Open Web Text)

Max Activating Token Index: 272

Text #13

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5078. Min Act: -0.1700

Data Index: 2786776 (Open Web Text)

Max Activating Token Index: 884

Text #14

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.5087. Min Act: -0.1700

Data Index: 348291 (Open Web Text)

Max Activating Token Index: 628

Text #15

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.4988. Min Act: -0.1700

Data Index: 4747471 (Open Web Text)

Max Activating Token Index: 891

Text #16

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.4906. Min Act: -0.1700

Data Index: 6314425 (Open Web Text)

Max Activating Token Index: 827

Text #17

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.4883. Min Act: -0.1700

Data Index: 449851 (Open Web Text)

Max Activating Token Index: 871

Text #18

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.4773. Min Act: -0.1700

Data Index: 8304097 (Open Web Text)

Max Activating Token Index: 387

Text #19

Max Range: 2.5555. Min Range: -2.5555

Max Act: 2.4578. Min Act: -0.1700

Data Index: 4587841 (Open Web Text)

Max Activating Token Index: 168