Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3016 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.6303. Min Act: -0.1700

Data Index: 7445107 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5965. Min Act: -0.1700

Data Index: 8508749 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5718. Min Act: -0.1700

Data Index: 4325773 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5377. Min Act: -0.1700

Data Index: 6702278 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5416. Min Act: -0.1700

Data Index: 3702586 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5463. Min Act: -0.1700

Data Index: 308358 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.5510. Min Act: -0.1700

Data Index: 8368902 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4542. Min Act: -0.1700

Data Index: 212366 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4843. Min Act: -0.1700

Data Index: 1315643 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4665. Min Act: -0.1700

Data Index: 4993310 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4919. Min Act: -0.1700

Data Index: 194242 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4433. Min Act: -0.1700

Data Index: 4043621 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4833. Min Act: -0.1693

Data Index: 1632776 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4819. Min Act: -0.1700

Data Index: 7704164 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4762. Min Act: -0.1700

Data Index: 7487720 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4970. Min Act: -0.1700

Data Index: 7459190 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4539. Min Act: -0.1700

Data Index: 2796126 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4788. Min Act: -0.1700

Data Index: 2205818 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.4544. Min Act: -0.1700

Data Index: 5941304 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 7.6303. Min Range: -7.6303

Max Act: 7.3849. Min Act: -0.1700

Data Index: 5235346 (Open Web Text)

Max Activating Token Index: 8