Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2781 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8454. Min Act: -0.1700

Data Index: 1841061 (Open Web Text)

Max Activating Token Index: 200

Text #1

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8518. Min Act: -0.1700

Data Index: 1978981 (Open Web Text)

Max Activating Token Index: 184

Text #2

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8625. Min Act: -0.1700

Data Index: 7698117 (Open Web Text)

Max Activating Token Index: 175

Text #3

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8571. Min Act: -0.1700

Data Index: 4695245 (Open Web Text)

Max Activating Token Index: 124

Text #4

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8485. Min Act: -0.1700

Data Index: 5848560 (Open Web Text)

Max Activating Token Index: 154

Text #5

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7975. Min Act: -0.1700

Data Index: 605517 (Open Web Text)

Max Activating Token Index: 123

Text #6

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8122. Min Act: -0.1700

Data Index: 8381638 (Open Web Text)

Max Activating Token Index: 681

Text #7

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8011. Min Act: -0.1700

Data Index: 7964697 (Open Web Text)

Max Activating Token Index: 218

Text #8

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7893. Min Act: -0.1700

Data Index: 7929662 (Open Web Text)

Max Activating Token Index: 118

Text #9

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8004. Min Act: -0.1700

Data Index: 7745823 (Open Web Text)

Max Activating Token Index: 250

Text #10

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8061. Min Act: -0.1700

Data Index: 6928870 (Open Web Text)

Max Activating Token Index: 255

Text #11

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8176. Min Act: -0.1700

Data Index: 6850768 (Open Web Text)

Max Activating Token Index: 543

Text #12

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7995. Min Act: -0.1700

Data Index: 6798167 (Open Web Text)

Max Activating Token Index: 956

Text #13

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7998. Min Act: -0.1700

Data Index: 6540707 (Open Web Text)

Max Activating Token Index: 533

Text #14

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7917. Min Act: -0.1700

Data Index: 6008758 (Open Web Text)

Max Activating Token Index: 379

Text #15

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8088. Min Act: -0.1700

Data Index: 5378888 (Open Web Text)

Max Activating Token Index: 87

Text #16

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8115. Min Act: -0.1700

Data Index: 4751502 (Open Web Text)

Max Activating Token Index: 643

Text #17

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.8141. Min Act: -0.1700

Data Index: 2388672 (Open Web Text)

Max Activating Token Index: 262

Text #18

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7754. Min Act: -0.1700

Data Index: 1535800 (Open Web Text)

Max Activating Token Index: 201

Text #19

Max Range: 3.8625. Min Range: -3.8625

Max Act: 3.7794. Min Act: -0.1700

Data Index: 4327131 (Open Web Text)

Max Activating Token Index: 1019