Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1289 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.9109. Min Act: -0.1700

Data Index: 1923889 (Open Web Text)

Max Activating Token Index: 272

Text #1

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8778. Min Act: -0.1700

Data Index: 3668616 (Open Web Text)

Max Activating Token Index: 434

Text #2

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8571. Min Act: -0.1700

Data Index: 171685 (Open Web Text)

Max Activating Token Index: 750

Text #3

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8399. Min Act: -0.1700

Data Index: 3513707 (Open Web Text)

Max Activating Token Index: 218

Text #4

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8463. Min Act: -0.1700

Data Index: 56651 (Open Web Text)

Max Activating Token Index: 87

Text #5

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8046. Min Act: -0.1700

Data Index: 6859131 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7759. Min Act: -0.1700

Data Index: 6347639 (Open Web Text)

Max Activating Token Index: 980

Text #7

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.8068. Min Act: -0.1700

Data Index: 92964 (Open Web Text)

Max Activating Token Index: 268

Text #8

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7953. Min Act: -0.1700

Data Index: 1557559 (Open Web Text)

Max Activating Token Index: 1016

Text #9

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7964. Min Act: -0.1700

Data Index: 4020564 (Open Web Text)

Max Activating Token Index: 171

Text #10

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7476. Min Act: -0.1700

Data Index: 2054815 (Open Web Text)

Max Activating Token Index: 522

Text #11

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7699. Min Act: -0.1700

Data Index: 4270476 (Open Web Text)

Max Activating Token Index: 947

Text #12

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7556. Min Act: -0.1700

Data Index: 619195 (Open Web Text)

Max Activating Token Index: 641

Text #13

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7570. Min Act: -0.1700

Data Index: 521861 (Open Web Text)

Max Activating Token Index: 739

Text #14

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7774. Min Act: -0.1700

Data Index: 2840047 (Open Web Text)

Max Activating Token Index: 217

Text #15

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7736. Min Act: -0.1700

Data Index: 3849735 (Open Web Text)

Max Activating Token Index: 856

Text #16

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7421. Min Act: -0.1700

Data Index: 2815539 (Open Web Text)

Max Activating Token Index: 225

Text #17

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7453. Min Act: -0.1700

Data Index: 5527144 (Open Web Text)

Max Activating Token Index: 181

Text #18

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7289. Min Act: -0.1700

Data Index: 3849733 (Open Web Text)

Max Activating Token Index: 682

Text #19

Max Range: 3.9109. Min Range: -3.9109

Max Act: 3.7305. Min Act: -0.1700

Data Index: 6579911 (Open Web Text)

Max Activating Token Index: 466