Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1338 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.6606. Min Act: -0.1700

Data Index: 5805291 (Open Web Text)

Max Activating Token Index: 963

Text #1

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.6824. Min Act: -0.1700

Data Index: 7778684 (Open Web Text)

Max Activating Token Index: 398

Text #2

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.6124. Min Act: -0.1700

Data Index: 6853439 (Open Web Text)

Max Activating Token Index: 1017

Text #3

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5942. Min Act: -0.1700

Data Index: 7489337 (Open Web Text)

Max Activating Token Index: 128

Text #4

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5829. Min Act: -0.1699

Data Index: 2493564 (Open Web Text)

Max Activating Token Index: 118

Text #5

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5194. Min Act: -0.1700

Data Index: 5964648 (Open Web Text)

Max Activating Token Index: 637

Text #6

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5705. Min Act: -0.1700

Data Index: 2198669 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5420. Min Act: -0.1700

Data Index: 2095333 (Open Web Text)

Max Activating Token Index: 669

Text #8

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5450. Min Act: -0.1700

Data Index: 1418879 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5396. Min Act: -0.1700

Data Index: 2833632 (Open Web Text)

Max Activating Token Index: 118

Text #10

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5202. Min Act: -0.1700

Data Index: 7095289 (Open Web Text)

Max Activating Token Index: 813

Text #11

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5423. Min Act: -0.1700

Data Index: 3049743 (Open Web Text)

Max Activating Token Index: 387

Text #12

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5011. Min Act: -0.1700

Data Index: 1021231 (Open Web Text)

Max Activating Token Index: 321

Text #13

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5526. Min Act: -0.1700

Data Index: 6949716 (Open Web Text)

Max Activating Token Index: 780

Text #14

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5107. Min Act: -0.1700

Data Index: 3346515 (Open Web Text)

Max Activating Token Index: 39

Text #15

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5109. Min Act: -0.1700

Data Index: 1161100 (Open Web Text)

Max Activating Token Index: 798

Text #16

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5382. Min Act: -0.1700

Data Index: 3856931 (Open Web Text)

Max Activating Token Index: 941

Text #17

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.5114. Min Act: -0.1700

Data Index: 3175835 (Open Web Text)

Max Activating Token Index: 25

Text #18

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.4976. Min Act: -0.1700

Data Index: 4967121 (Open Web Text)

Max Activating Token Index: 576

Text #19

Max Range: 1.6824. Min Range: -1.6824

Max Act: 1.4683. Min Act: -0.1700

Data Index: 762831 (Open Web Text)

Max Activating Token Index: 854