Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1837 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0293. Min Range: -6.0293

Max Act: 6.0293. Min Act: -0.1700

Data Index: 2690604 (Open Web Text)

Max Activating Token Index: 124

Text #1

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.9769. Min Act: -0.1700

Data Index: 7905139 (Open Web Text)

Max Activating Token Index: 148

Text #2

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.9839. Min Act: -0.1700

Data Index: 3865156 (Open Web Text)

Max Activating Token Index: 530

Text #3

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.9095. Min Act: -0.1700

Data Index: 4226718 (Open Web Text)

Max Activating Token Index: 1014

Text #4

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.9003. Min Act: -0.1700

Data Index: 2030226 (Open Web Text)

Max Activating Token Index: 59

Text #5

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.9285. Min Act: -0.1700

Data Index: 4932224 (Open Web Text)

Max Activating Token Index: 636

Text #6

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8832. Min Act: -0.1700

Data Index: 5047889 (Open Web Text)

Max Activating Token Index: 135

Text #7

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8978. Min Act: -0.1700

Data Index: 5472344 (Open Web Text)

Max Activating Token Index: 611

Text #8

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8911. Min Act: -0.1700

Data Index: 6805761 (Open Web Text)

Max Activating Token Index: 35

Text #9

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8788. Min Act: -0.1700

Data Index: 903329 (Open Web Text)

Max Activating Token Index: 183

Text #10

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8996. Min Act: -0.1700

Data Index: 7392284 (Open Web Text)

Max Activating Token Index: 117

Text #11

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8909. Min Act: -0.1700

Data Index: 1872531 (Open Web Text)

Max Activating Token Index: 122

Text #12

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8822. Min Act: -0.1700

Data Index: 3665571 (Open Web Text)

Max Activating Token Index: 773

Text #13

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8826. Min Act: -0.1700

Data Index: 4884530 (Open Web Text)

Max Activating Token Index: 503

Text #14

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8986. Min Act: -0.1700

Data Index: 4252583 (Open Web Text)

Max Activating Token Index: 80

Text #15

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8826. Min Act: -0.1700

Data Index: 1465732 (Open Web Text)

Max Activating Token Index: 1014

Text #16

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8773. Min Act: -0.1700

Data Index: 1505202 (Open Web Text)

Max Activating Token Index: 381

Text #17

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8542. Min Act: -0.1700

Data Index: 5374246 (Open Web Text)

Max Activating Token Index: 139

Text #18

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8748. Min Act: -0.1700

Data Index: 7583323 (Open Web Text)

Max Activating Token Index: 749

Text #19

Max Range: 6.0293. Min Range: -6.0293

Max Act: 5.8548. Min Act: -0.1700

Data Index: 299586 (Open Web Text)

Max Activating Token Index: 146