Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 639 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.9392. Min Act: -0.1700

Data Index: 6360479 (Open Web Text)

Max Activating Token Index: 127

Text #1

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.9256. Min Act: -0.1700

Data Index: 1006191 (Open Web Text)

Max Activating Token Index: 122

Text #2

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.7689. Min Act: -0.1700

Data Index: 1514326 (Open Web Text)

Max Activating Token Index: 769

Text #3

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.7137. Min Act: -0.1700

Data Index: 5354908 (Open Web Text)

Max Activating Token Index: 821

Text #4

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.6405. Min Act: -0.1700

Data Index: 8659249 (Open Web Text)

Max Activating Token Index: 491

Text #5

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.6113. Min Act: -0.1700

Data Index: 5181950 (Open Web Text)

Max Activating Token Index: 641

Text #6

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.5458. Min Act: -0.1700

Data Index: 3258652 (Open Web Text)

Max Activating Token Index: 377

Text #7

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.4919. Min Act: -0.1700

Data Index: 4684579 (Open Web Text)

Max Activating Token Index: 715

Text #8

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.4726. Min Act: -0.1700

Data Index: 7402920 (Open Web Text)

Max Activating Token Index: 903

Text #9

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.4303. Min Act: -0.1700

Data Index: 7352777 (Open Web Text)

Max Activating Token Index: 322

Text #10

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.4401. Min Act: -0.1700

Data Index: 5896628 (Open Web Text)

Max Activating Token Index: 963

Text #11

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3999. Min Act: -0.1700

Data Index: 4887138 (Open Web Text)

Max Activating Token Index: 234

Text #12

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3799. Min Act: -0.1700

Data Index: 387531 (Open Web Text)

Max Activating Token Index: 753

Text #13

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3921. Min Act: -0.1700

Data Index: 7668871 (Open Web Text)

Max Activating Token Index: 168

Text #14

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3766. Min Act: -0.1700

Data Index: 708272 (Open Web Text)

Max Activating Token Index: 260

Text #15

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3622. Min Act: -0.1700

Data Index: 4930420 (Open Web Text)

Max Activating Token Index: 481

Text #16

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3484. Min Act: -0.1700

Data Index: 351279 (Open Web Text)

Max Activating Token Index: 463

Text #17

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3555. Min Act: -0.1700

Data Index: 6137949 (Open Web Text)

Max Activating Token Index: 546

Text #18

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3668. Min Act: -0.1700

Data Index: 2512855 (Open Web Text)

Max Activating Token Index: 977

Text #19

Max Range: 4.9392. Min Range: -4.9392

Max Act: 4.3492. Min Act: -0.1700

Data Index: 6804011 (Open Web Text)

Max Activating Token Index: 697