Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2082 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.8512. Min Act: -0.1700

Data Index: 5171064 (Open Web Text)

Max Activating Token Index: 884

Text #1

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.8382. Min Act: -0.1700

Data Index: 1536551 (Open Web Text)

Max Activating Token Index: 958

Text #2

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.8138. Min Act: -0.1700

Data Index: 2923269 (Open Web Text)

Max Activating Token Index: 915

Text #3

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7822. Min Act: -0.1700

Data Index: 2537614 (Open Web Text)

Max Activating Token Index: 883

Text #4

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7654. Min Act: -0.1700

Data Index: 2240170 (Open Web Text)

Max Activating Token Index: 1014

Text #5

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7441. Min Act: -0.1700

Data Index: 6642319 (Open Web Text)

Max Activating Token Index: 871

Text #6

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7357. Min Act: -0.1700

Data Index: 7904980 (Open Web Text)

Max Activating Token Index: 657

Text #7

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7563. Min Act: -0.1700

Data Index: 7402660 (Open Web Text)

Max Activating Token Index: 932

Text #8

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7505. Min Act: -0.1700

Data Index: 6526201 (Open Web Text)

Max Activating Token Index: 797

Text #9

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7432. Min Act: -0.1700

Data Index: 1616110 (Open Web Text)

Max Activating Token Index: 927

Text #10

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7292. Min Act: -0.1700

Data Index: 876169 (Open Web Text)

Max Activating Token Index: 967

Text #11

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7066. Min Act: -0.1700

Data Index: 8353761 (Open Web Text)

Max Activating Token Index: 785

Text #12

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7176. Min Act: -0.1700

Data Index: 3999471 (Open Web Text)

Max Activating Token Index: 769

Text #13

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7362. Min Act: -0.1700

Data Index: 8787920 (Open Web Text)

Max Activating Token Index: 801

Text #14

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7159. Min Act: -0.1700

Data Index: 5122198 (Open Web Text)

Max Activating Token Index: 820

Text #15

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7020. Min Act: -0.1700

Data Index: 2790016 (Open Web Text)

Max Activating Token Index: 838

Text #16

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7070. Min Act: -0.1700

Data Index: 4197517 (Open Web Text)

Max Activating Token Index: 403

Text #17

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7329. Min Act: -0.1700

Data Index: 6472749 (Open Web Text)

Max Activating Token Index: 883

Text #18

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.7050. Min Act: -0.1700

Data Index: 2955825 (Open Web Text)

Max Activating Token Index: 896

Text #19

Max Range: 1.8512. Min Range: -1.8512

Max Act: 1.6952. Min Act: -0.1700

Data Index: 5073864 (Open Web Text)

Max Activating Token Index: 504