Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2570 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.6752. Min Act: -0.1700

Data Index: 4264475 (Open Web Text)

Max Activating Token Index: 147

Text #1

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.6171. Min Act: -0.1700

Data Index: 1212548 (Open Web Text)

Max Activating Token Index: 75

Text #2

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.6129. Min Act: -0.1700

Data Index: 6782784 (Open Web Text)

Max Activating Token Index: 485

Text #3

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.6098. Min Act: -0.1700

Data Index: 5776889 (Open Web Text)

Max Activating Token Index: 530

Text #4

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5943. Min Act: -0.1700

Data Index: 8799487 (Open Web Text)

Max Activating Token Index: 609

Text #5

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5898. Min Act: -0.1700

Data Index: 5028197 (Open Web Text)

Max Activating Token Index: 950

Text #6

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5741. Min Act: -0.1700

Data Index: 3627156 (Open Web Text)

Max Activating Token Index: 426

Text #7

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5884. Min Act: -0.1700

Data Index: 5873975 (Open Web Text)

Max Activating Token Index: 470

Text #8

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5737. Min Act: -0.1700

Data Index: 1541728 (Open Web Text)

Max Activating Token Index: 457

Text #9

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5750. Min Act: -0.1700

Data Index: 4995906 (Open Web Text)

Max Activating Token Index: 444

Text #10

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5695. Min Act: -0.1699

Data Index: 7988188 (Open Web Text)

Max Activating Token Index: 773

Text #11

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5659. Min Act: -0.1700

Data Index: 5806035 (Open Web Text)

Max Activating Token Index: 56

Text #12

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5536. Min Act: -0.1700

Data Index: 1411850 (Open Web Text)

Max Activating Token Index: 766

Text #13

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5640. Min Act: -0.1700

Data Index: 2757894 (Open Web Text)

Max Activating Token Index: 558

Text #14

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5603. Min Act: -0.1700

Data Index: 5114644 (Open Web Text)

Max Activating Token Index: 41

Text #15

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5696. Min Act: -0.1700

Data Index: 5704051 (Open Web Text)

Max Activating Token Index: 140

Text #16

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5696. Min Act: -0.1700

Data Index: 5476391 (Open Web Text)

Max Activating Token Index: 666

Text #17

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5711. Min Act: -0.1700

Data Index: 8354953 (Open Web Text)

Max Activating Token Index: 377

Text #18

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5715. Min Act: -0.1700

Data Index: 1505903 (Open Web Text)

Max Activating Token Index: 521

Text #19

Max Range: 1.6752. Min Range: -1.6752

Max Act: 1.5629. Min Act: -0.1700

Data Index: 127998 (Open Web Text)

Max Activating Token Index: 754