Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2226 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.8164. Min Act: -0.1700

Data Index: 8828694 (Open Web Text)

Max Activating Token Index: 306

Text #1

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.7073. Min Act: -0.1700

Data Index: 3885042 (Open Web Text)

Max Activating Token Index: 992

Text #2

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6827. Min Act: -0.1700

Data Index: 238955 (Open Web Text)

Max Activating Token Index: 910

Text #3

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6971. Min Act: -0.1700

Data Index: 1154839 (Open Web Text)

Max Activating Token Index: 943

Text #4

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6990. Min Act: -0.1700

Data Index: 7694223 (Open Web Text)

Max Activating Token Index: 521

Text #5

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6570. Min Act: -0.1700

Data Index: 7741239 (Open Web Text)

Max Activating Token Index: 268

Text #6

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6381. Min Act: -0.1700

Data Index: 1594097 (Open Web Text)

Max Activating Token Index: 169

Text #7

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6493. Min Act: -0.1700

Data Index: 2684245 (Open Web Text)

Max Activating Token Index: 83

Text #8

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6293. Min Act: -0.1700

Data Index: 5651146 (Open Web Text)

Max Activating Token Index: 141

Text #9

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6173. Min Act: -0.1700

Data Index: 5252248 (Open Web Text)

Max Activating Token Index: 55

Text #10

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5943. Min Act: -0.1700

Data Index: 6699798 (Open Web Text)

Max Activating Token Index: 506

Text #11

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.6316. Min Act: -0.1700

Data Index: 8479599 (Open Web Text)

Max Activating Token Index: 576

Text #12

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5993. Min Act: -0.1700

Data Index: 7171807 (Open Web Text)

Max Activating Token Index: 291

Text #13

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5930. Min Act: -0.1700

Data Index: 6169203 (Open Web Text)

Max Activating Token Index: 396

Text #14

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5918. Min Act: -0.1700

Data Index: 4803448 (Open Web Text)

Max Activating Token Index: 118

Text #15

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5158. Min Act: -0.1700

Data Index: 2672802 (Open Web Text)

Max Activating Token Index: 909

Text #16

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5714. Min Act: -0.1700

Data Index: 5056859 (Open Web Text)

Max Activating Token Index: 332

Text #17

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5511. Min Act: -0.1700

Data Index: 1889444 (Open Web Text)

Max Activating Token Index: 883

Text #18

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5646. Min Act: -0.1700

Data Index: 4926279 (Open Web Text)

Max Activating Token Index: 950

Text #19

Max Range: 3.8164. Min Range: -3.8164

Max Act: 3.5795. Min Act: -0.1700

Data Index: 1500576 (Open Web Text)

Max Activating Token Index: 407