Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1759 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.7382. Min Act: -0.1700

Data Index: 1166392 (Open Web Text)

Max Activating Token Index: 865

Text #1

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.7244. Min Act: -0.1700

Data Index: 4648046 (Open Web Text)

Max Activating Token Index: 935

Text #2

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.7330. Min Act: -0.1700

Data Index: 3727698 (Open Web Text)

Max Activating Token Index: 644

Text #3

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.7283. Min Act: -0.1700

Data Index: 2534503 (Open Web Text)

Max Activating Token Index: 66

Text #4

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6764. Min Act: -0.1700

Data Index: 8492374 (Open Web Text)

Max Activating Token Index: 652

Text #5

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6815. Min Act: -0.1700

Data Index: 5411825 (Open Web Text)

Max Activating Token Index: 580

Text #6

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6458. Min Act: -0.1700

Data Index: 8651164 (Open Web Text)

Max Activating Token Index: 487

Text #7

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6595. Min Act: -0.1700

Data Index: 8677865 (Open Web Text)

Max Activating Token Index: 512

Text #8

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6835. Min Act: -0.1700

Data Index: 248759 (Open Web Text)

Max Activating Token Index: 739

Text #9

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6379. Min Act: -0.1700

Data Index: 3946592 (Open Web Text)

Max Activating Token Index: 882

Text #10

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6216. Min Act: -0.1700

Data Index: 7337561 (Open Web Text)

Max Activating Token Index: 1006

Text #11

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.6247. Min Act: -0.1700

Data Index: 4052008 (Open Web Text)

Max Activating Token Index: 917

Text #12

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5935. Min Act: -0.1700

Data Index: 8121914 (Open Web Text)

Max Activating Token Index: 512

Text #13

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5899. Min Act: -0.1700

Data Index: 3574605 (Open Web Text)

Max Activating Token Index: 582

Text #14

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5777. Min Act: -0.1700

Data Index: 1712306 (Open Web Text)

Max Activating Token Index: 121

Text #15

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5743. Min Act: -0.1700

Data Index: 673629 (Open Web Text)

Max Activating Token Index: 684

Text #16

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5810. Min Act: -0.1700

Data Index: 4647947 (Open Web Text)

Max Activating Token Index: 652

Text #17

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5616. Min Act: -0.1700

Data Index: 711536 (Open Web Text)

Max Activating Token Index: 784

Text #18

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5640. Min Act: -0.1700

Data Index: 2052669 (Open Web Text)

Max Activating Token Index: 453

Text #19

Max Range: 3.7382. Min Range: -3.7382

Max Act: 3.5539. Min Act: -0.1700

Data Index: 5359086 (Open Web Text)

Max Activating Token Index: 786