Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 892 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.1914. Min Act: -0.1700

Data Index: 2432382 (Open Web Text)

Max Activating Token Index: 427

Text #1

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.1903. Min Act: -0.1700

Data Index: 5233227 (Open Web Text)

Max Activating Token Index: 615

Text #2

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0880. Min Act: -0.1700

Data Index: 3459262 (Open Web Text)

Max Activating Token Index: 606

Text #3

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0686. Min Act: -0.1700

Data Index: 4550902 (Open Web Text)

Max Activating Token Index: 970

Text #4

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0619. Min Act: -0.1700

Data Index: 3236593 (Open Web Text)

Max Activating Token Index: 865

Text #5

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0458. Min Act: -0.1700

Data Index: 6659913 (Open Web Text)

Max Activating Token Index: 101

Text #6

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0362. Min Act: -0.1700

Data Index: 4084531 (Open Web Text)

Max Activating Token Index: 67

Text #7

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9962. Min Act: -0.1700

Data Index: 2464119 (Open Web Text)

Max Activating Token Index: 78

Text #8

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0077. Min Act: -0.1700

Data Index: 3593650 (Open Web Text)

Max Activating Token Index: 805

Text #9

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0326. Min Act: -0.1700

Data Index: 5814695 (Open Web Text)

Max Activating Token Index: 883

Text #10

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0169. Min Act: -0.1700

Data Index: 7700772 (Open Web Text)

Max Activating Token Index: 998

Text #11

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9957. Min Act: -0.1700

Data Index: 7487610 (Open Web Text)

Max Activating Token Index: 939

Text #12

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9990. Min Act: -0.1700

Data Index: 474195 (Open Web Text)

Max Activating Token Index: 453

Text #13

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9643. Min Act: -0.1700

Data Index: 918874 (Open Web Text)

Max Activating Token Index: 692

Text #14

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0048. Min Act: -0.1700

Data Index: 4057856 (Open Web Text)

Max Activating Token Index: 139

Text #15

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9684. Min Act: -0.1700

Data Index: 1682658 (Open Web Text)

Max Activating Token Index: 509

Text #16

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9807. Min Act: -0.1700

Data Index: 8618902 (Open Web Text)

Max Activating Token Index: 705

Text #17

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9499. Min Act: -0.1700

Data Index: 4629504 (Open Web Text)

Max Activating Token Index: 919

Text #18

Max Range: 4.1914. Min Range: -4.1914

Max Act: 4.0013. Min Act: -0.1700

Data Index: 4577588 (Open Web Text)

Max Activating Token Index: 839

Text #19

Max Range: 4.1914. Min Range: -4.1914

Max Act: 3.9200. Min Act: -0.1700

Data Index: 4257835 (Open Web Text)

Max Activating Token Index: 609