Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2402 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0403. Min Range: -3.0403

Max Act: 3.0403. Min Act: -0.1700

Data Index: 6750151 (Open Web Text)

Max Activating Token Index: 953

Text #1

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9950. Min Act: -0.1700

Data Index: 7116368 (Open Web Text)

Max Activating Token Index: 907

Text #2

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9460. Min Act: -0.1700

Data Index: 7222696 (Open Web Text)

Max Activating Token Index: 665

Text #3

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9692. Min Act: -0.1700

Data Index: 4981437 (Open Web Text)

Max Activating Token Index: 458

Text #4

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9490. Min Act: -0.1700

Data Index: 5336087 (Open Web Text)

Max Activating Token Index: 407

Text #5

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9293. Min Act: -0.1700

Data Index: 5332567 (Open Web Text)

Max Activating Token Index: 727

Text #6

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8823. Min Act: -0.1700

Data Index: 5426784 (Open Web Text)

Max Activating Token Index: 688

Text #7

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9071. Min Act: -0.1700

Data Index: 3223583 (Open Web Text)

Max Activating Token Index: 1016

Text #8

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9020. Min Act: -0.1700

Data Index: 5302704 (Open Web Text)

Max Activating Token Index: 524

Text #9

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.9025. Min Act: -0.1700

Data Index: 8559363 (Open Web Text)

Max Activating Token Index: 166

Text #10

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8925. Min Act: -0.1700

Data Index: 6754774 (Open Web Text)

Max Activating Token Index: 453

Text #11

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8774. Min Act: -0.1700

Data Index: 7132846 (Open Web Text)

Max Activating Token Index: 371

Text #12

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8672. Min Act: -0.1700

Data Index: 7444410 (Open Web Text)

Max Activating Token Index: 452

Text #13

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8801. Min Act: -0.1700

Data Index: 4615310 (Open Web Text)

Max Activating Token Index: 911

Text #14

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8729. Min Act: -0.1700

Data Index: 7950318 (Open Web Text)

Max Activating Token Index: 950

Text #15

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8585. Min Act: -0.1700

Data Index: 1980863 (Open Web Text)

Max Activating Token Index: 342

Text #16

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8695. Min Act: -0.1700

Data Index: 2386228 (Open Web Text)

Max Activating Token Index: 500

Text #17

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8839. Min Act: -0.1700

Data Index: 3982295 (Open Web Text)

Max Activating Token Index: 204

Text #18

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8631. Min Act: -0.1700

Data Index: 3237857 (Open Web Text)

Max Activating Token Index: 307

Text #19

Max Range: 3.0403. Min Range: -3.0403

Max Act: 2.8547. Min Act: -0.1700

Data Index: 2337494 (Open Web Text)

Max Activating Token Index: 375