Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2499 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.7013. Min Act: -0.1700

Data Index: 5362751 (Open Web Text)

Max Activating Token Index: 257

Text #1

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6975. Min Act: -0.1700

Data Index: 5551273 (Open Web Text)

Max Activating Token Index: 186

Text #2

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6489. Min Act: -0.1700

Data Index: 3655583 (Open Web Text)

Max Activating Token Index: 986

Text #3

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6402. Min Act: -0.1700

Data Index: 3655584 (Open Web Text)

Max Activating Token Index: 21

Text #4

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6121. Min Act: -0.1700

Data Index: 861354 (Open Web Text)

Max Activating Token Index: 368

Text #5

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6071. Min Act: -0.1700

Data Index: 1674788 (Open Web Text)

Max Activating Token Index: 681

Text #6

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6052. Min Act: -0.1700

Data Index: 3855847 (Open Web Text)

Max Activating Token Index: 890

Text #7

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6015. Min Act: -0.1700

Data Index: 5499345 (Open Web Text)

Max Activating Token Index: 853

Text #8

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5935. Min Act: -0.1700

Data Index: 1674791 (Open Web Text)

Max Activating Token Index: 613

Text #9

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5561. Min Act: -0.1700

Data Index: 3899140 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5574. Min Act: -0.1700

Data Index: 7424503 (Open Web Text)

Max Activating Token Index: 448

Text #11

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5475. Min Act: -0.1700

Data Index: 5948957 (Open Web Text)

Max Activating Token Index: 498

Text #12

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5757. Min Act: -0.1700

Data Index: 1992965 (Open Web Text)

Max Activating Token Index: 890

Text #13

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5673. Min Act: -0.1700

Data Index: 4961451 (Open Web Text)

Max Activating Token Index: 766

Text #14

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5717. Min Act: -0.1700

Data Index: 7874025 (Open Web Text)

Max Activating Token Index: 366

Text #15

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5730. Min Act: -0.1700

Data Index: 5966395 (Open Web Text)

Max Activating Token Index: 224

Text #16

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5326. Min Act: -0.1700

Data Index: 2023500 (Open Web Text)

Max Activating Token Index: 947

Text #17

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5483. Min Act: -0.1700

Data Index: 2223569 (Open Web Text)

Max Activating Token Index: 652

Text #18

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5341. Min Act: -0.1700

Data Index: 6261498 (Open Web Text)

Max Activating Token Index: 188

Text #19

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5329. Min Act: -0.1700

Data Index: 8067945 (Open Web Text)

Max Activating Token Index: 527