Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 158 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.4726. Min Act: -0.1700

Data Index: 7569386 (Open Web Text)

Max Activating Token Index: 804

Text #1

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.4280. Min Act: -0.1700

Data Index: 3430752 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.4303. Min Act: -0.1700

Data Index: 6189453 (Open Web Text)

Max Activating Token Index: 836

Text #3

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.4074. Min Act: -0.1700

Data Index: 5752399 (Open Web Text)

Max Activating Token Index: 57

Text #4

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3922. Min Act: -0.1700

Data Index: 5606901 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3907. Min Act: -0.1700

Data Index: 6955003 (Open Web Text)

Max Activating Token Index: 425

Text #6

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3759. Min Act: -0.1700

Data Index: 3441595 (Open Web Text)

Max Activating Token Index: 236

Text #7

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3832. Min Act: -0.1700

Data Index: 5000292 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3809. Min Act: -0.1700

Data Index: 1695965 (Open Web Text)

Max Activating Token Index: 290

Text #9

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3908. Min Act: -0.1700

Data Index: 4679840 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3924. Min Act: -0.1700

Data Index: 839397 (Open Web Text)

Max Activating Token Index: 421

Text #11

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3863. Min Act: -0.1700

Data Index: 2612558 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3600. Min Act: -0.1700

Data Index: 4402335 (Open Web Text)

Max Activating Token Index: 18

Text #13

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3566. Min Act: -0.1700

Data Index: 8569859 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3667. Min Act: -0.1700

Data Index: 7146620 (Open Web Text)

Max Activating Token Index: 664

Text #15

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3697. Min Act: -0.1700

Data Index: 6147265 (Open Web Text)

Max Activating Token Index: 30

Text #16

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3647. Min Act: -0.1700

Data Index: 5134350 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3742. Min Act: -0.1700

Data Index: 2564499 (Open Web Text)

Max Activating Token Index: 904

Text #18

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3404. Min Act: -0.1700

Data Index: 4403505 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 2.4726. Min Range: -2.4726

Max Act: 2.3520. Min Act: -0.1700

Data Index: 7038044 (Open Web Text)

Max Activating Token Index: 229