Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3044 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0804. Min Range: -3.0804

Max Act: 3.0804. Min Act: -0.1700

Data Index: 5431183 (Open Web Text)

Max Activating Token Index: 380

Text #1

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9967. Min Act: -0.1700

Data Index: 2683918 (Open Web Text)

Max Activating Token Index: 255

Text #2

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9698. Min Act: -0.1700

Data Index: 1745402 (Open Web Text)

Max Activating Token Index: 224

Text #3

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9702. Min Act: -0.1700

Data Index: 1441690 (Open Web Text)

Max Activating Token Index: 910

Text #4

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9115. Min Act: -0.1700

Data Index: 5879331 (Open Web Text)

Max Activating Token Index: 199

Text #5

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9011. Min Act: -0.1700

Data Index: 3302884 (Open Web Text)

Max Activating Token Index: 263

Text #6

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.9052. Min Act: -0.1700

Data Index: 4128890 (Open Web Text)

Max Activating Token Index: 380

Text #7

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8752. Min Act: -0.1700

Data Index: 4362124 (Open Web Text)

Max Activating Token Index: 69

Text #8

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8479. Min Act: -0.1700

Data Index: 6138172 (Open Web Text)

Max Activating Token Index: 962

Text #9

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8474. Min Act: -0.1700

Data Index: 604653 (Open Web Text)

Max Activating Token Index: 56

Text #10

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8502. Min Act: -0.1700

Data Index: 735558 (Open Web Text)

Max Activating Token Index: 758

Text #11

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8477. Min Act: -0.1700

Data Index: 4544493 (Open Web Text)

Max Activating Token Index: 402

Text #12

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8275. Min Act: -0.1700

Data Index: 444639 (Open Web Text)

Max Activating Token Index: 799

Text #13

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8323. Min Act: -0.1700

Data Index: 8469859 (Open Web Text)

Max Activating Token Index: 645

Text #14

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8304. Min Act: -0.1700

Data Index: 2325697 (Open Web Text)

Max Activating Token Index: 412

Text #15

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8137. Min Act: -0.1700

Data Index: 440868 (Open Web Text)

Max Activating Token Index: 690

Text #16

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.8168. Min Act: -0.1700

Data Index: 735560 (Open Web Text)

Max Activating Token Index: 548

Text #17

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.7787. Min Act: -0.1700

Data Index: 3544607 (Open Web Text)

Max Activating Token Index: 431

Text #18

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.7948. Min Act: -0.1700

Data Index: 4768002 (Open Web Text)

Max Activating Token Index: 534

Text #19

Max Range: 3.0804. Min Range: -3.0804

Max Act: 2.7834. Min Act: -0.1700

Data Index: 4247637 (Open Web Text)

Max Activating Token Index: 468