Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2009 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.2590. Min Act: -0.1700

Data Index: 3990250 (Open Web Text)

Max Activating Token Index: 766

Text #1

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.2000. Min Act: -0.1700

Data Index: 7442529 (Open Web Text)

Max Activating Token Index: 572

Text #2

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.2868. Min Act: -0.1700

Data Index: 1207551 (Open Web Text)

Max Activating Token Index: 846

Text #3

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.1939. Min Act: -0.1700

Data Index: 3698581 (Open Web Text)

Max Activating Token Index: 1007

Text #4

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.1704. Min Act: -0.1700

Data Index: 7418278 (Open Web Text)

Max Activating Token Index: 606

Text #5

Max Range: 5.2868. Min Range: -5.2868

Max Act: 5.0299. Min Act: -0.1700

Data Index: 4348880 (Open Web Text)

Max Activating Token Index: 198

Text #6

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9655. Min Act: -0.1700

Data Index: 3729025 (Open Web Text)

Max Activating Token Index: 598

Text #7

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9267. Min Act: -0.1700

Data Index: 6607007 (Open Web Text)

Max Activating Token Index: 224

Text #8

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9101. Min Act: -0.1700

Data Index: 3011410 (Open Web Text)

Max Activating Token Index: 541

Text #9

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9401. Min Act: -0.1700

Data Index: 3523884 (Open Web Text)

Max Activating Token Index: 76

Text #10

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9532. Min Act: -0.1700

Data Index: 8050328 (Open Web Text)

Max Activating Token Index: 841

Text #11

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8768. Min Act: -0.1700

Data Index: 3242346 (Open Web Text)

Max Activating Token Index: 52

Text #12

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.9069. Min Act: -0.1700

Data Index: 4098029 (Open Web Text)

Max Activating Token Index: 314

Text #13

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8880. Min Act: -0.1700

Data Index: 7753727 (Open Web Text)

Max Activating Token Index: 504

Text #14

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8995. Min Act: -0.1700

Data Index: 4747478 (Open Web Text)

Max Activating Token Index: 975

Text #15

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8680. Min Act: -0.1700

Data Index: 957525 (Open Web Text)

Max Activating Token Index: 679

Text #16

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8638. Min Act: -0.1700

Data Index: 2116029 (Open Web Text)

Max Activating Token Index: 629

Text #17

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8764. Min Act: -0.1700

Data Index: 1969202 (Open Web Text)

Max Activating Token Index: 871

Text #18

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8771. Min Act: -0.1700

Data Index: 1529983 (Open Web Text)

Max Activating Token Index: 615

Text #19

Max Range: 5.2868. Min Range: -5.2868

Max Act: 4.8372. Min Act: -0.1700

Data Index: 6376403 (Open Web Text)

Max Activating Token Index: 208