Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1887 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.6681. Min Act: -0.1700

Data Index: 7826707 (Open Web Text)

Max Activating Token Index: 235

Text #1

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.5847. Min Act: -0.1700

Data Index: 3780659 (Open Web Text)

Max Activating Token Index: 23

Text #2

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.5512. Min Act: -0.1700

Data Index: 877816 (Open Web Text)

Max Activating Token Index: 239

Text #3

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.5018. Min Act: -0.1700

Data Index: 5908975 (Open Web Text)

Max Activating Token Index: 81

Text #4

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.5044. Min Act: -0.1700

Data Index: 8595709 (Open Web Text)

Max Activating Token Index: 70

Text #5

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4115. Min Act: -0.1635

Data Index: 136761 (Open Web Text)

Max Activating Token Index: 560

Text #6

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4292. Min Act: -0.1588

Data Index: 827026 (Open Web Text)

Max Activating Token Index: 340

Text #7

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4347. Min Act: -0.1700

Data Index: 1163590 (Open Web Text)

Max Activating Token Index: 309

Text #8

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4367. Min Act: -0.1690

Data Index: 1811769 (Open Web Text)

Max Activating Token Index: 551

Text #9

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4043. Min Act: -0.1651

Data Index: 1811775 (Open Web Text)

Max Activating Token Index: 407

Text #10

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4074. Min Act: -0.1611

Data Index: 6839020 (Open Web Text)

Max Activating Token Index: 448

Text #11

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.4178. Min Act: -0.1700

Data Index: 2391242 (Open Web Text)

Max Activating Token Index: 489

Text #12

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3866. Min Act: -0.1694

Data Index: 1811776 (Open Web Text)

Max Activating Token Index: 594

Text #13

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3976. Min Act: -0.1655

Data Index: 6839024 (Open Web Text)

Max Activating Token Index: 419

Text #14

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3769. Min Act: -0.1692

Data Index: 3821089 (Open Web Text)

Max Activating Token Index: 704

Text #15

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3671. Min Act: -0.1700

Data Index: 4337773 (Open Web Text)

Max Activating Token Index: 198

Text #16

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3711. Min Act: -0.1671

Data Index: 6631871 (Open Web Text)

Max Activating Token Index: 695

Text #17

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3683. Min Act: -0.1651

Data Index: 4611094 (Open Web Text)

Max Activating Token Index: 713

Text #18

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3688. Min Act: -0.1024

Data Index: 6631872 (Open Web Text)

Max Activating Token Index: 834

Text #19

Max Range: 2.6681. Min Range: -2.6681

Max Act: 2.3708. Min Act: -0.1679

Data Index: 1811774 (Open Web Text)

Max Activating Token Index: 412