Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 553 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.6595. Min Act: -0.1700

Data Index: 3606188 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.6073. Min Act: -0.1700

Data Index: 4791649 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.6061. Min Act: -0.1700

Data Index: 6990605 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5996. Min Act: -0.1700

Data Index: 5770151 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5778. Min Act: -0.1700

Data Index: 5103516 (Open Web Text)

Max Activating Token Index: 13

Text #5

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5831. Min Act: -0.1700

Data Index: 354542 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5811. Min Act: -0.1700

Data Index: 7202140 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5704. Min Act: -0.1700

Data Index: 5642092 (Open Web Text)

Max Activating Token Index: 419

Text #8

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5708. Min Act: -0.1700

Data Index: 6725454 (Open Web Text)

Max Activating Token Index: 9

Text #9

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5666. Min Act: -0.1700

Data Index: 2942095 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5605. Min Act: -0.1700

Data Index: 1443557 (Open Web Text)

Max Activating Token Index: 663

Text #11

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5586. Min Act: -0.1700

Data Index: 2796477 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5580. Min Act: -0.1700

Data Index: 6651211 (Open Web Text)

Max Activating Token Index: 47

Text #13

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5512. Min Act: -0.1700

Data Index: 4381122 (Open Web Text)

Max Activating Token Index: 11

Text #14

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5641. Min Act: -0.1700

Data Index: 6034632 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5635. Min Act: -0.1700

Data Index: 4327828 (Open Web Text)

Max Activating Token Index: 9

Text #16

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5623. Min Act: -0.1700

Data Index: 4085653 (Open Web Text)

Max Activating Token Index: 37

Text #17

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5597. Min Act: -0.1700

Data Index: 2542771 (Open Web Text)

Max Activating Token Index: 15

Text #18

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5700. Min Act: -0.1700

Data Index: 2452536 (Open Web Text)

Max Activating Token Index: 203

Text #19

Max Range: 2.6595. Min Range: -2.6595

Max Act: 2.5608. Min Act: -0.1700

Data Index: 5737382 (Open Web Text)

Max Activating Token Index: 9