Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2049 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5991. Min Act: -0.0957

Data Index: 2355413 (Open Web Text)

Max Activating Token Index: 1022

Text #1

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5893. Min Act: -0.0985

Data Index: 2073042 (Open Web Text)

Max Activating Token Index: 1022

Text #2

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5846. Min Act: -0.1074

Data Index: 8023175 (Open Web Text)

Max Activating Token Index: 1022

Text #3

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5777. Min Act: -0.0928

Data Index: 389794 (Open Web Text)

Max Activating Token Index: 1022

Text #4

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5619. Min Act: -0.0983

Data Index: 6503282 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5440. Min Act: -0.0353

Data Index: 5503955 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5381. Min Act: -0.1020

Data Index: 1609534 (Open Web Text)

Max Activating Token Index: 1022

Text #7

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5432. Min Act: -0.1008

Data Index: 7360023 (Open Web Text)

Max Activating Token Index: 1020

Text #8

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5408. Min Act: -0.1216

Data Index: 5406425 (Open Web Text)

Max Activating Token Index: 1016

Text #9

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5382. Min Act: -0.0680

Data Index: 663436 (Open Web Text)

Max Activating Token Index: 1022

Text #10

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5354. Min Act: -0.0827

Data Index: 5304010 (Open Web Text)

Max Activating Token Index: 1021

Text #11

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5348. Min Act: -0.0325

Data Index: 5503958 (Open Web Text)

Max Activating Token Index: 1021

Text #12

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5331. Min Act: -0.0945

Data Index: 2049725 (Open Web Text)

Max Activating Token Index: 1022

Text #13

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5299. Min Act: -0.0815

Data Index: 8229985 (Open Web Text)

Max Activating Token Index: 1022

Text #14

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5308. Min Act: -0.0980

Data Index: 1191908 (Open Web Text)

Max Activating Token Index: 1022

Text #15

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5269. Min Act: -0.0705

Data Index: 7087254 (Open Web Text)

Max Activating Token Index: 1012

Text #16

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5231. Min Act: -0.0935

Data Index: 8229982 (Open Web Text)

Max Activating Token Index: 1022

Text #17

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5218. Min Act: -0.1025

Data Index: 8229981 (Open Web Text)

Max Activating Token Index: 1021

Text #18

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5243. Min Act: -0.0591

Data Index: 8229995 (Open Web Text)

Max Activating Token Index: 1022

Text #19

Max Range: 0.5991. Min Range: -0.5991

Max Act: 0.5184. Min Act: -0.0764

Data Index: 663437 (Open Web Text)

Max Activating Token Index: 1022