Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 455 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5889. Min Act: -0.0196

Data Index: 6066728 (Open Web Text)

Max Activating Token Index: 1020

Text #1

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5754. Min Act: 0.1340

Data Index: 3353496 (Open Web Text)

Max Activating Token Index: 1006

Text #2

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5743. Min Act: -0.1083

Data Index: 8571245 (Open Web Text)

Max Activating Token Index: 1014

Text #3

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5754. Min Act: -0.0391

Data Index: 6123123 (Open Web Text)

Max Activating Token Index: 1019

Text #4

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5762. Min Act: 0.2404

Data Index: 3353498 (Open Web Text)

Max Activating Token Index: 1005

Text #5

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5703. Min Act: 0.2404

Data Index: 3353493 (Open Web Text)

Max Activating Token Index: 1021

Text #6

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5678. Min Act: 0.0161

Data Index: 1725395 (Open Web Text)

Max Activating Token Index: 1014

Text #7

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5628. Min Act: -0.0010

Data Index: 5703943 (Open Web Text)

Max Activating Token Index: 1021

Text #8

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5626. Min Act: -0.0590

Data Index: 1766822 (Open Web Text)

Max Activating Token Index: 1021

Text #9

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5598. Min Act: -0.0764

Data Index: 3692373 (Open Web Text)

Max Activating Token Index: 1009

Text #10

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5609. Min Act: 0.0177

Data Index: 2691845 (Open Web Text)

Max Activating Token Index: 1019

Text #11

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5583. Min Act: 0.2404

Data Index: 3353500 (Open Web Text)

Max Activating Token Index: 1021

Text #12

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5565. Min Act: 0.0054

Data Index: 3353495 (Open Web Text)

Max Activating Token Index: 1005

Text #13

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5500. Min Act: -0.0841

Data Index: 7982816 (Open Web Text)

Max Activating Token Index: 1021

Text #14

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5522. Min Act: 0.0136

Data Index: 5855459 (Open Web Text)

Max Activating Token Index: 1020

Text #15

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5453. Min Act: 0.0150

Data Index: 523415 (Open Web Text)

Max Activating Token Index: 1019

Text #16

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5432. Min Act: -0.0781

Data Index: 1286876 (Open Web Text)

Max Activating Token Index: 1014

Text #17

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5425. Min Act: 0.0341

Data Index: 4412270 (Open Web Text)

Max Activating Token Index: 1008

Text #18

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5454. Min Act: 0.0134

Data Index: 2756594 (Open Web Text)

Max Activating Token Index: 1020

Text #19

Max Range: 1.5889. Min Range: -1.5889

Max Act: 1.5394. Min Act: 0.0172

Data Index: 3353487 (Open Web Text)

Max Activating Token Index: 1021