Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1093 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1719. Min Range: -5.1719

Max Act: 5.1516. Min Act: -0.1700

Data Index: 4035109 (Open Web Text)

Max Activating Token Index: 508

Text #1

Max Range: 5.1719. Min Range: -5.1719

Max Act: 5.1719. Min Act: -0.1700

Data Index: 2439881 (Open Web Text)

Max Activating Token Index: 891

Text #2

Max Range: 5.1719. Min Range: -5.1719

Max Act: 5.0900. Min Act: -0.1700

Data Index: 4035115 (Open Web Text)

Max Activating Token Index: 522

Text #3

Max Range: 5.1719. Min Range: -5.1719

Max Act: 5.1122. Min Act: -0.1700

Data Index: 5282999 (Open Web Text)

Max Activating Token Index: 831

Text #4

Max Range: 5.1719. Min Range: -5.1719

Max Act: 5.0000. Min Act: -0.1700

Data Index: 8174840 (Open Web Text)

Max Activating Token Index: 488

Text #5

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.9552. Min Act: -0.1700

Data Index: 3132688 (Open Web Text)

Max Activating Token Index: 55

Text #6

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.9387. Min Act: -0.1700

Data Index: 2307991 (Open Web Text)

Max Activating Token Index: 275

Text #7

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.9180. Min Act: -0.1700

Data Index: 4776779 (Open Web Text)

Max Activating Token Index: 196

Text #8

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.9365. Min Act: -0.1700

Data Index: 7328438 (Open Web Text)

Max Activating Token Index: 760

Text #9

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8718. Min Act: -0.1700

Data Index: 5497360 (Open Web Text)

Max Activating Token Index: 330

Text #10

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8933. Min Act: -0.1700

Data Index: 5735745 (Open Web Text)

Max Activating Token Index: 922

Text #11

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8070. Min Act: -0.1700

Data Index: 6521342 (Open Web Text)

Max Activating Token Index: 284

Text #12

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.7816. Min Act: -0.1700

Data Index: 929080 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8216. Min Act: -0.1700

Data Index: 2504735 (Open Web Text)

Max Activating Token Index: 802

Text #14

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8237. Min Act: -0.1700

Data Index: 982962 (Open Web Text)

Max Activating Token Index: 310

Text #15

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8098. Min Act: -0.1700

Data Index: 323006 (Open Web Text)

Max Activating Token Index: 247

Text #16

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8029. Min Act: -0.1700

Data Index: 2034728 (Open Web Text)

Max Activating Token Index: 655

Text #17

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.7950. Min Act: -0.1700

Data Index: 5358258 (Open Web Text)

Max Activating Token Index: 234

Text #18

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8227. Min Act: -0.1700

Data Index: 6216463 (Open Web Text)

Max Activating Token Index: 148

Text #19

Max Range: 5.1719. Min Range: -5.1719

Max Act: 4.8269. Min Act: -0.1700

Data Index: 6604087 (Open Web Text)

Max Activating Token Index: 442