Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3062 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.4013. Min Act: -0.1700

Data Index: 5606211 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.3686. Min Act: -0.1700

Data Index: 4424663 (Open Web Text)

Max Activating Token Index: 310

Text #2

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.2605. Min Act: -0.1700

Data Index: 1312580 (Open Web Text)

Max Activating Token Index: 758

Text #3

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.1617. Min Act: -0.1700

Data Index: 7333126 (Open Web Text)

Max Activating Token Index: 236

Text #4

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.1166. Min Act: -0.1700

Data Index: 7401587 (Open Web Text)

Max Activating Token Index: 209

Text #5

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.1147. Min Act: -0.1700

Data Index: 8713001 (Open Web Text)

Max Activating Token Index: 692

Text #6

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0911. Min Act: -0.1700

Data Index: 2803426 (Open Web Text)

Max Activating Token Index: 548

Text #7

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0956. Min Act: -0.1700

Data Index: 2210349 (Open Web Text)

Max Activating Token Index: 643

Text #8

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0949. Min Act: -0.1700

Data Index: 7600756 (Open Web Text)

Max Activating Token Index: 356

Text #9

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0649. Min Act: -0.1700

Data Index: 7342282 (Open Web Text)

Max Activating Token Index: 666

Text #10

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0508. Min Act: -0.1699

Data Index: 891563 (Open Web Text)

Max Activating Token Index: 964

Text #11

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0395. Min Act: -0.1700

Data Index: 7963690 (Open Web Text)

Max Activating Token Index: 521

Text #12

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0053. Min Act: -0.1700

Data Index: 5278567 (Open Web Text)

Max Activating Token Index: 498

Text #13

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0227. Min Act: -0.1700

Data Index: 4979584 (Open Web Text)

Max Activating Token Index: 406

Text #14

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0126. Min Act: -0.1700

Data Index: 7065815 (Open Web Text)

Max Activating Token Index: 251

Text #15

Max Range: 5.4013. Min Range: -5.4013

Max Act: 4.9973. Min Act: -0.1700

Data Index: 6239143 (Open Web Text)

Max Activating Token Index: 687

Text #16

Max Range: 5.4013. Min Range: -5.4013

Max Act: 4.9815. Min Act: -0.1700

Data Index: 2949118 (Open Web Text)

Max Activating Token Index: 953

Text #17

Max Range: 5.4013. Min Range: -5.4013

Max Act: 4.9848. Min Act: -0.1700

Data Index: 3398213 (Open Web Text)

Max Activating Token Index: 579

Text #18

Max Range: 5.4013. Min Range: -5.4013

Max Act: 5.0160. Min Act: -0.1700

Data Index: 749718 (Open Web Text)

Max Activating Token Index: 337

Text #19

Max Range: 5.4013. Min Range: -5.4013

Max Act: 4.9613. Min Act: -0.1700

Data Index: 6057790 (Open Web Text)

Max Activating Token Index: 728