Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1217 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.4421. Min Act: -0.1700

Data Index: 3230300 (Open Web Text)

Max Activating Token Index: 437

Text #1

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.4355. Min Act: -0.1700

Data Index: 6959184 (Open Web Text)

Max Activating Token Index: 75

Text #2

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.3373. Min Act: -0.1700

Data Index: 1263219 (Open Web Text)

Max Activating Token Index: 857

Text #3

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.3294. Min Act: -0.1700

Data Index: 8484783 (Open Web Text)

Max Activating Token Index: 347

Text #4

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.3479. Min Act: -0.1700

Data Index: 24892 (Open Web Text)

Max Activating Token Index: 525

Text #5

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.3250. Min Act: -0.1700

Data Index: 6577202 (Open Web Text)

Max Activating Token Index: 326

Text #6

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2929. Min Act: -0.1700

Data Index: 8434681 (Open Web Text)

Max Activating Token Index: 747

Text #7

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2868. Min Act: -0.1700

Data Index: 8157612 (Open Web Text)

Max Activating Token Index: 802

Text #8

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2754. Min Act: -0.1700

Data Index: 517990 (Open Web Text)

Max Activating Token Index: 587

Text #9

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2858. Min Act: -0.1700

Data Index: 8789382 (Open Web Text)

Max Activating Token Index: 1013

Text #10

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2155. Min Act: -0.1700

Data Index: 4980721 (Open Web Text)

Max Activating Token Index: 538

Text #11

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2489. Min Act: -0.1700

Data Index: 7265823 (Open Web Text)

Max Activating Token Index: 791

Text #12

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2439. Min Act: -0.1700

Data Index: 1700221 (Open Web Text)

Max Activating Token Index: 235

Text #13

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2226. Min Act: -0.1700

Data Index: 1798003 (Open Web Text)

Max Activating Token Index: 659

Text #14

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2021. Min Act: -0.1700

Data Index: 1241816 (Open Web Text)

Max Activating Token Index: 506

Text #15

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2434. Min Act: -0.1700

Data Index: 665729 (Open Web Text)

Max Activating Token Index: 583

Text #16

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2128. Min Act: -0.1700

Data Index: 3868869 (Open Web Text)

Max Activating Token Index: 609

Text #17

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2393. Min Act: -0.1700

Data Index: 3643647 (Open Web Text)

Max Activating Token Index: 657

Text #18

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2070. Min Act: -0.1700

Data Index: 6262818 (Open Web Text)

Max Activating Token Index: 909

Text #19

Max Range: 4.4421. Min Range: -4.4421

Max Act: 4.2252. Min Act: -0.1700

Data Index: 490096 (Open Web Text)

Max Activating Token Index: 90