Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 126 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2755. Min Act: -0.1700

Data Index: 8017865 (Open Web Text)

Max Activating Token Index: 404

Text #1

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2820. Min Act: -0.1700

Data Index: 4881313 (Open Web Text)

Max Activating Token Index: 155

Text #2

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2175. Min Act: -0.1700

Data Index: 2682500 (Open Web Text)

Max Activating Token Index: 220

Text #3

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2293. Min Act: -0.1700

Data Index: 228331 (Open Web Text)

Max Activating Token Index: 350

Text #4

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1988. Min Act: -0.1700

Data Index: 3036127 (Open Web Text)

Max Activating Token Index: 293

Text #5

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2027. Min Act: -0.1700

Data Index: 7821899 (Open Web Text)

Max Activating Token Index: 733

Text #6

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1940. Min Act: -0.1700

Data Index: 406710 (Open Web Text)

Max Activating Token Index: 167

Text #7

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2057. Min Act: -0.1700

Data Index: 7218714 (Open Web Text)

Max Activating Token Index: 463

Text #8

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2229. Min Act: -0.1700

Data Index: 5031810 (Open Web Text)

Max Activating Token Index: 774

Text #9

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1841. Min Act: -0.1700

Data Index: 7795594 (Open Web Text)

Max Activating Token Index: 130

Text #10

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1878. Min Act: -0.1700

Data Index: 4046264 (Open Web Text)

Max Activating Token Index: 247

Text #11

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.2009. Min Act: -0.1700

Data Index: 739848 (Open Web Text)

Max Activating Token Index: 821

Text #12

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1902. Min Act: -0.1700

Data Index: 6340473 (Open Web Text)

Max Activating Token Index: 572

Text #13

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1854. Min Act: -0.1700

Data Index: 2681901 (Open Web Text)

Max Activating Token Index: 218

Text #14

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1752. Min Act: -0.1700

Data Index: 7640669 (Open Web Text)

Max Activating Token Index: 269

Text #15

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1723. Min Act: -0.1700

Data Index: 4538465 (Open Web Text)

Max Activating Token Index: 812

Text #16

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1879. Min Act: -0.1700

Data Index: 7104173 (Open Web Text)

Max Activating Token Index: 1017

Text #17

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1620. Min Act: -0.1700

Data Index: 7029007 (Open Web Text)

Max Activating Token Index: 359

Text #18

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1668. Min Act: -0.1700

Data Index: 3475469 (Open Web Text)

Max Activating Token Index: 187

Text #19

Max Range: 3.2820. Min Range: -3.2820

Max Act: 3.1591. Min Act: -0.1700

Data Index: 6387558 (Open Web Text)

Max Activating Token Index: 239