Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 593 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6655. Min Act: -0.1700

Data Index: 4169850 (Open Web Text)

Max Activating Token Index: 309

Text #1

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6514. Min Act: -0.1697

Data Index: 6741315 (Open Web Text)

Max Activating Token Index: 1006

Text #2

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6197. Min Act: -0.1700

Data Index: 5237047 (Open Web Text)

Max Activating Token Index: 372

Text #3

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6231. Min Act: -0.1700

Data Index: 6649285 (Open Web Text)

Max Activating Token Index: 158

Text #4

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6067. Min Act: -0.1700

Data Index: 1972318 (Open Web Text)

Max Activating Token Index: 948

Text #5

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6025. Min Act: -0.1700

Data Index: 3483203 (Open Web Text)

Max Activating Token Index: 672

Text #6

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.6106. Min Act: -0.1700

Data Index: 1259432 (Open Web Text)

Max Activating Token Index: 415

Text #7

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5933. Min Act: -0.1700

Data Index: 107849 (Open Web Text)

Max Activating Token Index: 486

Text #8

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5981. Min Act: -0.1700

Data Index: 1287365 (Open Web Text)

Max Activating Token Index: 1012

Text #9

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5975. Min Act: -0.1700

Data Index: 2969853 (Open Web Text)

Max Activating Token Index: 294

Text #10

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5908. Min Act: -0.1700

Data Index: 670859 (Open Web Text)

Max Activating Token Index: 994

Text #11

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5848. Min Act: -0.1700

Data Index: 3212798 (Open Web Text)

Max Activating Token Index: 543

Text #12

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5843. Min Act: -0.1700

Data Index: 459495 (Open Web Text)

Max Activating Token Index: 412

Text #13

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5838. Min Act: -0.1699

Data Index: 282576 (Open Web Text)

Max Activating Token Index: 333

Text #14

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5843. Min Act: -0.1700

Data Index: 5753889 (Open Web Text)

Max Activating Token Index: 632

Text #15

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5815. Min Act: -0.1700

Data Index: 5951473 (Open Web Text)

Max Activating Token Index: 902

Text #16

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5828. Min Act: -0.1700

Data Index: 7359022 (Open Web Text)

Max Activating Token Index: 639

Text #17

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5729. Min Act: -0.1700

Data Index: 7441189 (Open Web Text)

Max Activating Token Index: 662

Text #18

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5773. Min Act: -0.1700

Data Index: 6149723 (Open Web Text)

Max Activating Token Index: 294

Text #19

Max Range: 1.6655. Min Range: -1.6655

Max Act: 1.5636. Min Act: -0.1697

Data Index: 4607637 (Open Web Text)

Max Activating Token Index: 302