Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2086 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5290. Min Act: -0.1700

Data Index: 4686635 (Open Web Text)

Max Activating Token Index: 776

Text #1

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5128. Min Act: -0.1700

Data Index: 2801089 (Open Web Text)

Max Activating Token Index: 434

Text #2

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5090. Min Act: -0.1700

Data Index: 1891653 (Open Web Text)

Max Activating Token Index: 82

Text #3

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5194. Min Act: -0.1700

Data Index: 6787149 (Open Web Text)

Max Activating Token Index: 469

Text #4

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5312. Min Act: -0.1700

Data Index: 5571506 (Open Web Text)

Max Activating Token Index: 493

Text #5

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4953. Min Act: -0.1700

Data Index: 905301 (Open Web Text)

Max Activating Token Index: 978

Text #6

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4937. Min Act: -0.1700

Data Index: 4991606 (Open Web Text)

Max Activating Token Index: 496

Text #7

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4673. Min Act: -0.1700

Data Index: 1006687 (Open Web Text)

Max Activating Token Index: 788

Text #8

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.5058. Min Act: -0.1700

Data Index: 6785913 (Open Web Text)

Max Activating Token Index: 607

Text #9

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4989. Min Act: -0.1700

Data Index: 549671 (Open Web Text)

Max Activating Token Index: 682

Text #10

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4551. Min Act: -0.1700

Data Index: 995163 (Open Web Text)

Max Activating Token Index: 990

Text #11

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4006. Min Act: -0.1700

Data Index: 1837085 (Open Web Text)

Max Activating Token Index: 571

Text #12

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4490. Min Act: -0.1699

Data Index: 6142621 (Open Web Text)

Max Activating Token Index: 411

Text #13

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4395. Min Act: -0.1700

Data Index: 3843191 (Open Web Text)

Max Activating Token Index: 784

Text #14

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4429. Min Act: -0.1700

Data Index: 5262255 (Open Web Text)

Max Activating Token Index: 619

Text #15

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4363. Min Act: -0.1700

Data Index: 8638608 (Open Web Text)

Max Activating Token Index: 935

Text #16

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4709. Min Act: -0.1700

Data Index: 3574115 (Open Web Text)

Max Activating Token Index: 351

Text #17

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4406. Min Act: -0.1700

Data Index: 6997683 (Open Web Text)

Max Activating Token Index: 104

Text #18

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4072. Min Act: -0.1700

Data Index: 7472525 (Open Web Text)

Max Activating Token Index: 993

Text #19

Max Range: 4.5312. Min Range: -4.5312

Max Act: 4.4308. Min Act: -0.1700

Data Index: 816183 (Open Web Text)

Max Activating Token Index: 194