Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1538 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0717. Min Range: -4.0717

Max Act: 4.0717. Min Act: -0.1700

Data Index: 5561775 (Open Web Text)

Max Activating Token Index: 547

Text #1

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8788. Min Act: -0.1700

Data Index: 3481834 (Open Web Text)

Max Activating Token Index: 156

Text #2

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8764. Min Act: -0.1700

Data Index: 2066984 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8220. Min Act: -0.1700

Data Index: 4010004 (Open Web Text)

Max Activating Token Index: 402

Text #4

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7855. Min Act: -0.1700

Data Index: 5165521 (Open Web Text)

Max Activating Token Index: 908

Text #5

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8019. Min Act: -0.1700

Data Index: 3179950 (Open Web Text)

Max Activating Token Index: 241

Text #6

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8135. Min Act: -0.1700

Data Index: 8572212 (Open Web Text)

Max Activating Token Index: 211

Text #7

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.8046. Min Act: -0.1700

Data Index: 8739344 (Open Web Text)

Max Activating Token Index: 839

Text #8

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7645. Min Act: -0.1700

Data Index: 4100116 (Open Web Text)

Max Activating Token Index: 427

Text #9

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7795. Min Act: -0.1700

Data Index: 464249 (Open Web Text)

Max Activating Token Index: 589

Text #10

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7631. Min Act: -0.1700

Data Index: 5984623 (Open Web Text)

Max Activating Token Index: 771

Text #11

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7042. Min Act: -0.1700

Data Index: 5271910 (Open Web Text)

Max Activating Token Index: 886

Text #12

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7537. Min Act: -0.1700

Data Index: 1563680 (Open Web Text)

Max Activating Token Index: 430

Text #13

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7350. Min Act: -0.1700

Data Index: 3045842 (Open Web Text)

Max Activating Token Index: 69

Text #14

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7410. Min Act: -0.1700

Data Index: 8171701 (Open Web Text)

Max Activating Token Index: 177

Text #15

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7113. Min Act: -0.1700

Data Index: 7286771 (Open Web Text)

Max Activating Token Index: 307

Text #16

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7233. Min Act: -0.1700

Data Index: 2871292 (Open Web Text)

Max Activating Token Index: 428

Text #17

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7007. Min Act: -0.1700

Data Index: 1486909 (Open Web Text)

Max Activating Token Index: 999

Text #18

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.7016. Min Act: -0.1700

Data Index: 2553219 (Open Web Text)

Max Activating Token Index: 496

Text #19

Max Range: 4.0717. Min Range: -4.0717

Max Act: 3.6902. Min Act: -0.1700

Data Index: 7582640 (Open Web Text)

Max Activating Token Index: 849