Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1097 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.6373. Min Act: -0.1700

Data Index: 7571040 (Open Web Text)

Max Activating Token Index: 835

Text #1

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.6632. Min Act: -0.1700

Data Index: 5715516 (Open Web Text)

Max Activating Token Index: 464

Text #2

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.6190. Min Act: -0.1700

Data Index: 240496 (Open Web Text)

Max Activating Token Index: 461

Text #3

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5726. Min Act: -0.1700

Data Index: 5128363 (Open Web Text)

Max Activating Token Index: 763

Text #4

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.6143. Min Act: -0.1700

Data Index: 23439 (Open Web Text)

Max Activating Token Index: 784

Text #5

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5355. Min Act: -0.1700

Data Index: 1994990 (Open Web Text)

Max Activating Token Index: 541

Text #6

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5705. Min Act: -0.1700

Data Index: 6506679 (Open Web Text)

Max Activating Token Index: 988

Text #7

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5596. Min Act: -0.1700

Data Index: 2106028 (Open Web Text)

Max Activating Token Index: 619

Text #8

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5259. Min Act: -0.1700

Data Index: 8296693 (Open Web Text)

Max Activating Token Index: 463

Text #9

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5364. Min Act: -0.1700

Data Index: 3425783 (Open Web Text)

Max Activating Token Index: 125

Text #10

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5184. Min Act: -0.1700

Data Index: 6122511 (Open Web Text)

Max Activating Token Index: 1006

Text #11

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.5250. Min Act: -0.1700

Data Index: 6351062 (Open Web Text)

Max Activating Token Index: 493

Text #12

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4883. Min Act: -0.1700

Data Index: 8338918 (Open Web Text)

Max Activating Token Index: 629

Text #13

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4779. Min Act: -0.1700

Data Index: 5298980 (Open Web Text)

Max Activating Token Index: 953

Text #14

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4707. Min Act: -0.1700

Data Index: 168617 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4685. Min Act: -0.1700

Data Index: 1609030 (Open Web Text)

Max Activating Token Index: 843

Text #16

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4596. Min Act: -0.1700

Data Index: 2007038 (Open Web Text)

Max Activating Token Index: 80

Text #17

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4191. Min Act: -0.1700

Data Index: 3029125 (Open Web Text)

Max Activating Token Index: 629

Text #18

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4398. Min Act: -0.1700

Data Index: 3776013 (Open Web Text)

Max Activating Token Index: 164

Text #19

Max Range: 3.6632. Min Range: -3.6632

Max Act: 3.4348. Min Act: -0.1700

Data Index: 2972246 (Open Web Text)

Max Activating Token Index: 751