Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 412 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.4470. Min Act: -0.1700

Data Index: 3332952 (Open Web Text)

Max Activating Token Index: 368

Text #1

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.2867. Min Act: -0.1700

Data Index: 5341441 (Open Web Text)

Max Activating Token Index: 353

Text #2

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.2687. Min Act: -0.1700

Data Index: 198014 (Open Web Text)

Max Activating Token Index: 511

Text #3

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.2068. Min Act: -0.1700

Data Index: 5939694 (Open Web Text)

Max Activating Token Index: 366

Text #4

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.1744. Min Act: -0.1700

Data Index: 6910303 (Open Web Text)

Max Activating Token Index: 111

Text #5

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.1080. Min Act: -0.1700

Data Index: 4646734 (Open Web Text)

Max Activating Token Index: 299

Text #6

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.0192. Min Act: -0.1700

Data Index: 7436592 (Open Web Text)

Max Activating Token Index: 316

Text #7

Max Range: 3.4470. Min Range: -3.4470

Max Act: 3.0476. Min Act: -0.1700

Data Index: 4582137 (Open Web Text)

Max Activating Token Index: 356

Text #8

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9792. Min Act: -0.1700

Data Index: 5509075 (Open Web Text)

Max Activating Token Index: 725

Text #9

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9362. Min Act: -0.1700

Data Index: 5360268 (Open Web Text)

Max Activating Token Index: 250

Text #10

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9108. Min Act: -0.1700

Data Index: 8609168 (Open Web Text)

Max Activating Token Index: 369

Text #11

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9028. Min Act: -0.1700

Data Index: 4449313 (Open Web Text)

Max Activating Token Index: 224

Text #12

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9536. Min Act: -0.1700

Data Index: 1953236 (Open Web Text)

Max Activating Token Index: 449

Text #13

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9127. Min Act: -0.1700

Data Index: 253131 (Open Web Text)

Max Activating Token Index: 682

Text #14

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.9065. Min Act: -0.1700

Data Index: 1596699 (Open Web Text)

Max Activating Token Index: 950

Text #15

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.8697. Min Act: -0.1700

Data Index: 30146 (Open Web Text)

Max Activating Token Index: 432

Text #16

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.8792. Min Act: -0.1700

Data Index: 2782857 (Open Web Text)

Max Activating Token Index: 724

Text #17

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.8742. Min Act: -0.1700

Data Index: 529670 (Open Web Text)

Max Activating Token Index: 669

Text #18

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.8380. Min Act: -0.1700

Data Index: 576276 (Open Web Text)

Max Activating Token Index: 279

Text #19

Max Range: 3.4470. Min Range: -3.4470

Max Act: 2.8413. Min Act: -0.1700

Data Index: 7323501 (Open Web Text)

Max Activating Token Index: 447