Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1123 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.6120. Min Act: -0.1700

Data Index: 3222479 (Open Web Text)

Max Activating Token Index: 26

Text #1

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.6171. Min Act: -0.1700

Data Index: 3702933 (Open Web Text)

Max Activating Token Index: 685

Text #2

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5969. Min Act: -0.1700

Data Index: 7031552 (Open Web Text)

Max Activating Token Index: 48

Text #3

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5610. Min Act: -0.1700

Data Index: 1058487 (Open Web Text)

Max Activating Token Index: 42

Text #4

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5783. Min Act: -0.1700

Data Index: 3755913 (Open Web Text)

Max Activating Token Index: 22

Text #5

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5724. Min Act: -0.1700

Data Index: 5638740 (Open Web Text)

Max Activating Token Index: 1016

Text #6

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5476. Min Act: -0.1700

Data Index: 709644 (Open Web Text)

Max Activating Token Index: 108

Text #7

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5362. Min Act: -0.1700

Data Index: 1731888 (Open Web Text)

Max Activating Token Index: 227

Text #8

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5518. Min Act: -0.1700

Data Index: 3627226 (Open Web Text)

Max Activating Token Index: 669

Text #9

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5488. Min Act: -0.1700

Data Index: 5045231 (Open Web Text)

Max Activating Token Index: 519

Text #10

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5321. Min Act: -0.1700

Data Index: 3642405 (Open Web Text)

Max Activating Token Index: 511

Text #11

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.4753. Min Act: -0.1700

Data Index: 2683788 (Open Web Text)

Max Activating Token Index: 924

Text #12

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5150. Min Act: -0.1700

Data Index: 6971749 (Open Web Text)

Max Activating Token Index: 168

Text #13

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5215. Min Act: -0.1700

Data Index: 8602192 (Open Web Text)

Max Activating Token Index: 133

Text #14

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5466. Min Act: -0.1700

Data Index: 7879593 (Open Web Text)

Max Activating Token Index: 884

Text #15

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.4923. Min Act: -0.1700

Data Index: 5344009 (Open Web Text)

Max Activating Token Index: 15

Text #16

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5438. Min Act: -0.1700

Data Index: 3608779 (Open Web Text)

Max Activating Token Index: 601

Text #17

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5067. Min Act: -0.1700

Data Index: 6731673 (Open Web Text)

Max Activating Token Index: 35

Text #18

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.5217. Min Act: -0.1700

Data Index: 713961 (Open Web Text)

Max Activating Token Index: 440

Text #19

Max Range: 3.6171. Min Range: -3.6171

Max Act: 3.4820. Min Act: -0.1700

Data Index: 484937 (Open Web Text)

Max Activating Token Index: 51