Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 166 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.6330. Min Act: -0.1700

Data Index: 5579649 (Open Web Text)

Max Activating Token Index: 270

Text #1

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.6185. Min Act: -0.1700

Data Index: 7681640 (Open Web Text)

Max Activating Token Index: 850

Text #2

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.5380. Min Act: -0.1700

Data Index: 6353966 (Open Web Text)

Max Activating Token Index: 923

Text #3

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.5452. Min Act: -0.1700

Data Index: 1138049 (Open Web Text)

Max Activating Token Index: 502

Text #4

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.5649. Min Act: -0.1700

Data Index: 3639384 (Open Web Text)

Max Activating Token Index: 643

Text #5

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4922. Min Act: -0.1700

Data Index: 3776718 (Open Web Text)

Max Activating Token Index: 252

Text #6

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4835. Min Act: -0.1700

Data Index: 2052126 (Open Web Text)

Max Activating Token Index: 770

Text #7

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4764. Min Act: -0.1700

Data Index: 5276332 (Open Web Text)

Max Activating Token Index: 724

Text #8

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4795. Min Act: -0.1700

Data Index: 5363548 (Open Web Text)

Max Activating Token Index: 498

Text #9

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4359. Min Act: -0.1700

Data Index: 8288528 (Open Web Text)

Max Activating Token Index: 158

Text #10

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4017. Min Act: -0.1700

Data Index: 551396 (Open Web Text)

Max Activating Token Index: 347

Text #11

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3974. Min Act: -0.1700

Data Index: 2366412 (Open Web Text)

Max Activating Token Index: 108

Text #12

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4250. Min Act: -0.1700

Data Index: 482181 (Open Web Text)

Max Activating Token Index: 734

Text #13

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3681. Min Act: -0.1700

Data Index: 6282562 (Open Web Text)

Max Activating Token Index: 212

Text #14

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3784. Min Act: -0.1700

Data Index: 2103302 (Open Web Text)

Max Activating Token Index: 506

Text #15

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3739. Min Act: -0.1700

Data Index: 5757611 (Open Web Text)

Max Activating Token Index: 864

Text #16

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3824. Min Act: -0.1700

Data Index: 2108899 (Open Web Text)

Max Activating Token Index: 869

Text #17

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.3917. Min Act: -0.1700

Data Index: 4429106 (Open Web Text)

Max Activating Token Index: 773

Text #18

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4097. Min Act: -0.1700

Data Index: 3986403 (Open Web Text)

Max Activating Token Index: 500

Text #19

Max Range: 4.6330. Min Range: -4.6330

Max Act: 4.4127. Min Act: -0.1700

Data Index: 2598598 (Open Web Text)

Max Activating Token Index: 747