Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1766 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.5318. Min Act: -0.1700

Data Index: 4767024 (Open Web Text)

Max Activating Token Index: 989

Text #1

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.5249. Min Act: -0.1700

Data Index: 8211464 (Open Web Text)

Max Activating Token Index: 837

Text #2

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.5193. Min Act: -0.1700

Data Index: 2153908 (Open Web Text)

Max Activating Token Index: 941

Text #3

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.5245. Min Act: -0.1700

Data Index: 7984119 (Open Web Text)

Max Activating Token Index: 809

Text #4

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.5293. Min Act: -0.1700

Data Index: 8111796 (Open Web Text)

Max Activating Token Index: 1009

Text #5

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4863. Min Act: -0.1700

Data Index: 3500476 (Open Web Text)

Max Activating Token Index: 579

Text #6

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4522. Min Act: -0.1700

Data Index: 5670602 (Open Web Text)

Max Activating Token Index: 1002

Text #7

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4629. Min Act: -0.1700

Data Index: 5319518 (Open Web Text)

Max Activating Token Index: 659

Text #8

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4454. Min Act: -0.1700

Data Index: 4276142 (Open Web Text)

Max Activating Token Index: 692

Text #9

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4305. Min Act: -0.1700

Data Index: 4224107 (Open Web Text)

Max Activating Token Index: 930

Text #10

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4188. Min Act: -0.1700

Data Index: 7893270 (Open Web Text)

Max Activating Token Index: 820

Text #11

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4042. Min Act: -0.1700

Data Index: 5669307 (Open Web Text)

Max Activating Token Index: 986

Text #12

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4168. Min Act: -0.1700

Data Index: 3152862 (Open Web Text)

Max Activating Token Index: 890

Text #13

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4115. Min Act: -0.1700

Data Index: 6409410 (Open Web Text)

Max Activating Token Index: 800

Text #14

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.3893. Min Act: -0.1700

Data Index: 8320220 (Open Web Text)

Max Activating Token Index: 532

Text #15

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4075. Min Act: -0.1700

Data Index: 1192649 (Open Web Text)

Max Activating Token Index: 214

Text #16

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4153. Min Act: -0.1700

Data Index: 8407410 (Open Web Text)

Max Activating Token Index: 706

Text #17

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.3938. Min Act: -0.1700

Data Index: 5482544 (Open Web Text)

Max Activating Token Index: 994

Text #18

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4032. Min Act: -0.1700

Data Index: 4254695 (Open Web Text)

Max Activating Token Index: 764

Text #19

Max Range: 3.5318. Min Range: -3.5318

Max Act: 3.4041. Min Act: -0.1700

Data Index: 7091848 (Open Web Text)

Max Activating Token Index: 661