Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2750 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.3099. Min Act: -0.1700

Data Index: 6006727 (Open Web Text)

Max Activating Token Index: 153

Text #1

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.1380. Min Act: -0.1585

Data Index: 7291296 (Open Web Text)

Max Activating Token Index: 187

Text #2

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.1258. Min Act: -0.1700

Data Index: 8669359 (Open Web Text)

Max Activating Token Index: 159

Text #3

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.1010. Min Act: -0.1693

Data Index: 8814229 (Open Web Text)

Max Activating Token Index: 114

Text #4

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.0906. Min Act: -0.1700

Data Index: 2427021 (Open Web Text)

Max Activating Token Index: 191

Text #5

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.0217. Min Act: -0.1700

Data Index: 1079491 (Open Web Text)

Max Activating Token Index: 927

Text #6

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.0633. Min Act: -0.1696

Data Index: 1441321 (Open Web Text)

Max Activating Token Index: 146

Text #7

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.0456. Min Act: -0.1634

Data Index: 1486864 (Open Web Text)

Max Activating Token Index: 129

Text #8

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9842. Min Act: -0.1700

Data Index: 2427016 (Open Web Text)

Max Activating Token Index: 249

Text #9

Max Range: 3.3099. Min Range: -3.3099

Max Act: 3.0062. Min Act: -0.1700

Data Index: 7165477 (Open Web Text)

Max Activating Token Index: 190

Text #10

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9896. Min Act: -0.1700

Data Index: 6852974 (Open Web Text)

Max Activating Token Index: 195

Text #11

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9635. Min Act: -0.1700

Data Index: 7783693 (Open Web Text)

Max Activating Token Index: 444

Text #12

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9647. Min Act: -0.1700

Data Index: 4058063 (Open Web Text)

Max Activating Token Index: 227

Text #13

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9489. Min Act: -0.1700

Data Index: 4057892 (Open Web Text)

Max Activating Token Index: 158

Text #14

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9523. Min Act: -0.1700

Data Index: 1656749 (Open Web Text)

Max Activating Token Index: 186

Text #15

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9608. Min Act: -0.1700

Data Index: 4020854 (Open Web Text)

Max Activating Token Index: 174

Text #16

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9698. Min Act: -0.1700

Data Index: 3488998 (Open Web Text)

Max Activating Token Index: 181

Text #17

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9227. Min Act: -0.1698

Data Index: 2914802 (Open Web Text)

Max Activating Token Index: 580

Text #18

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9495. Min Act: -0.1700

Data Index: 7087729 (Open Web Text)

Max Activating Token Index: 337

Text #19

Max Range: 3.3099. Min Range: -3.3099

Max Act: 2.9589. Min Act: -0.1700

Data Index: 1656765 (Open Web Text)

Max Activating Token Index: 169