Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1443 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0500. Min Range: -5.0500

Max Act: 5.0500. Min Act: -0.1700

Data Index: 1889494 (Open Web Text)

Max Activating Token Index: 113

Text #1

Max Range: 5.0500. Min Range: -5.0500

Max Act: 5.0201. Min Act: -0.1700

Data Index: 4028356 (Open Web Text)

Max Activating Token Index: 694

Text #2

Max Range: 5.0500. Min Range: -5.0500

Max Act: 5.0077. Min Act: -0.1700

Data Index: 2186649 (Open Web Text)

Max Activating Token Index: 768

Text #3

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.9964. Min Act: -0.1700

Data Index: 3054888 (Open Web Text)

Max Activating Token Index: 1020

Text #4

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8822. Min Act: -0.1700

Data Index: 5926158 (Open Web Text)

Max Activating Token Index: 153

Text #5

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.9012. Min Act: -0.1700

Data Index: 1425068 (Open Web Text)

Max Activating Token Index: 635

Text #6

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8913. Min Act: -0.1700

Data Index: 1867710 (Open Web Text)

Max Activating Token Index: 250

Text #7

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8935. Min Act: -0.1700

Data Index: 7147908 (Open Web Text)

Max Activating Token Index: 714

Text #8

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8673. Min Act: -0.1700

Data Index: 6257109 (Open Web Text)

Max Activating Token Index: 773

Text #9

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8505. Min Act: -0.1700

Data Index: 514074 (Open Web Text)

Max Activating Token Index: 730

Text #10

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8858. Min Act: -0.1700

Data Index: 7327331 (Open Web Text)

Max Activating Token Index: 884

Text #11

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8037. Min Act: -0.1700

Data Index: 2269702 (Open Web Text)

Max Activating Token Index: 461

Text #12

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8380. Min Act: -0.1700

Data Index: 4303254 (Open Web Text)

Max Activating Token Index: 67

Text #13

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8707. Min Act: -0.1700

Data Index: 1652594 (Open Web Text)

Max Activating Token Index: 406

Text #14

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8283. Min Act: -0.1700

Data Index: 6496353 (Open Web Text)

Max Activating Token Index: 420

Text #15

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8124. Min Act: -0.1700

Data Index: 72612 (Open Web Text)

Max Activating Token Index: 504

Text #16

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8028. Min Act: -0.1700

Data Index: 4525780 (Open Web Text)

Max Activating Token Index: 832

Text #17

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8016. Min Act: -0.1700

Data Index: 829508 (Open Web Text)

Max Activating Token Index: 837

Text #18

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8414. Min Act: -0.1700

Data Index: 3179591 (Open Web Text)

Max Activating Token Index: 964

Text #19

Max Range: 5.0500. Min Range: -5.0500

Max Act: 4.8087. Min Act: -0.1700

Data Index: 6789243 (Open Web Text)

Max Activating Token Index: 618