Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2543 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6663. Min Act: -0.1700

Data Index: 4658327 (Open Web Text)

Max Activating Token Index: 496

Text #1

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6555. Min Act: -0.1700

Data Index: 5028479 (Open Web Text)

Max Activating Token Index: 503

Text #2

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6328. Min Act: -0.1700

Data Index: 8315101 (Open Web Text)

Max Activating Token Index: 493

Text #3

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6311. Min Act: -0.1700

Data Index: 8604732 (Open Web Text)

Max Activating Token Index: 453

Text #4

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6288. Min Act: -0.1700

Data Index: 7616533 (Open Web Text)

Max Activating Token Index: 461

Text #5

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.6205. Min Act: -0.1700

Data Index: 7392422 (Open Web Text)

Max Activating Token Index: 476

Text #6

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5904. Min Act: -0.1700

Data Index: 2143776 (Open Web Text)

Max Activating Token Index: 445

Text #7

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5864. Min Act: -0.1700

Data Index: 4116964 (Open Web Text)

Max Activating Token Index: 508

Text #8

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5828. Min Act: -0.1700

Data Index: 4799920 (Open Web Text)

Max Activating Token Index: 526

Text #9

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5767. Min Act: -0.1700

Data Index: 7887341 (Open Web Text)

Max Activating Token Index: 466

Text #10

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5570. Min Act: -0.1700

Data Index: 700767 (Open Web Text)

Max Activating Token Index: 454

Text #11

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5661. Min Act: -0.1700

Data Index: 4444639 (Open Web Text)

Max Activating Token Index: 463

Text #12

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5723. Min Act: -0.1700

Data Index: 2685775 (Open Web Text)

Max Activating Token Index: 496

Text #13

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5509. Min Act: -0.1700

Data Index: 2814012 (Open Web Text)

Max Activating Token Index: 421

Text #14

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5598. Min Act: -0.1700

Data Index: 6504599 (Open Web Text)

Max Activating Token Index: 467

Text #15

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5575. Min Act: -0.1700

Data Index: 8132332 (Open Web Text)

Max Activating Token Index: 445

Text #16

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5581. Min Act: -0.1700

Data Index: 118142 (Open Web Text)

Max Activating Token Index: 498

Text #17

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5328. Min Act: -0.1700

Data Index: 2684250 (Open Web Text)

Max Activating Token Index: 420

Text #18

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5451. Min Act: -0.1700

Data Index: 8489226 (Open Web Text)

Max Activating Token Index: 466

Text #19

Max Range: 0.6663. Min Range: -0.6663

Max Act: 0.5581. Min Act: -0.1700

Data Index: 2209056 (Open Web Text)

Max Activating Token Index: 468