Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2737 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.5142. Min Act: -0.1700

Data Index: 8050455 (Open Web Text)

Max Activating Token Index: 629

Text #1

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.5063. Min Act: -0.1700

Data Index: 8405133 (Open Web Text)

Max Activating Token Index: 671

Text #2

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.5223. Min Act: -0.1700

Data Index: 1854113 (Open Web Text)

Max Activating Token Index: 806

Text #3

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4790. Min Act: -0.1700

Data Index: 7546460 (Open Web Text)

Max Activating Token Index: 258

Text #4

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4957. Min Act: -0.1700

Data Index: 7138606 (Open Web Text)

Max Activating Token Index: 731

Text #5

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4848. Min Act: -0.1700

Data Index: 6763065 (Open Web Text)

Max Activating Token Index: 662

Text #6

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4811. Min Act: -0.1700

Data Index: 6006743 (Open Web Text)

Max Activating Token Index: 672

Text #7

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.5009. Min Act: -0.1700

Data Index: 577019 (Open Web Text)

Max Activating Token Index: 649

Text #8

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4793. Min Act: -0.1700

Data Index: 5217182 (Open Web Text)

Max Activating Token Index: 757

Text #9

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.5058. Min Act: -0.1700

Data Index: 3436901 (Open Web Text)

Max Activating Token Index: 830

Text #10

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4873. Min Act: -0.1700

Data Index: 3132028 (Open Web Text)

Max Activating Token Index: 811

Text #11

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4940. Min Act: -0.1700

Data Index: 2562346 (Open Web Text)

Max Activating Token Index: 751

Text #12

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4519. Min Act: -0.1700

Data Index: 3886088 (Open Web Text)

Max Activating Token Index: 951

Text #13

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4471. Min Act: -0.1700

Data Index: 5322039 (Open Web Text)

Max Activating Token Index: 671

Text #14

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4668. Min Act: -0.1700

Data Index: 1420638 (Open Web Text)

Max Activating Token Index: 613

Text #15

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4752. Min Act: -0.1700

Data Index: 5491453 (Open Web Text)

Max Activating Token Index: 932

Text #16

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4557. Min Act: -0.1700

Data Index: 359120 (Open Web Text)

Max Activating Token Index: 691

Text #17

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4545. Min Act: -0.1700

Data Index: 3780635 (Open Web Text)

Max Activating Token Index: 228

Text #18

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4522. Min Act: -0.1700

Data Index: 6069877 (Open Web Text)

Max Activating Token Index: 650

Text #19

Max Range: 4.5223. Min Range: -4.5223

Max Act: 4.4503. Min Act: -0.1700

Data Index: 7061857 (Open Web Text)

Max Activating Token Index: 735