Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1965 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5656. Min Act: -0.1700

Data Index: 7294140 (Open Web Text)

Max Activating Token Index: 894

Text #1

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.6021. Min Act: -0.1700

Data Index: 3382324 (Open Web Text)

Max Activating Token Index: 362

Text #2

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5630. Min Act: -0.1700

Data Index: 2538095 (Open Web Text)

Max Activating Token Index: 383

Text #3

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5160. Min Act: -0.1700

Data Index: 1296218 (Open Web Text)

Max Activating Token Index: 379

Text #4

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5071. Min Act: -0.1700

Data Index: 1402379 (Open Web Text)

Max Activating Token Index: 113

Text #5

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5159. Min Act: -0.1700

Data Index: 4591787 (Open Web Text)

Max Activating Token Index: 73

Text #6

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5540. Min Act: -0.1700

Data Index: 5341560 (Open Web Text)

Max Activating Token Index: 146

Text #7

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5019. Min Act: -0.1700

Data Index: 3420959 (Open Web Text)

Max Activating Token Index: 550

Text #8

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.5033. Min Act: -0.1700

Data Index: 8295617 (Open Web Text)

Max Activating Token Index: 609

Text #9

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4846. Min Act: -0.1700

Data Index: 2076869 (Open Web Text)

Max Activating Token Index: 131

Text #10

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4714. Min Act: -0.1700

Data Index: 2697886 (Open Web Text)

Max Activating Token Index: 142

Text #11

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4523. Min Act: -0.1700

Data Index: 625193 (Open Web Text)

Max Activating Token Index: 379

Text #12

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4608. Min Act: -0.1700

Data Index: 5045472 (Open Web Text)

Max Activating Token Index: 330

Text #13

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4912. Min Act: -0.1700

Data Index: 591747 (Open Web Text)

Max Activating Token Index: 475

Text #14

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4694. Min Act: -0.1700

Data Index: 2478270 (Open Web Text)

Max Activating Token Index: 927

Text #15

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4533. Min Act: -0.1700

Data Index: 321199 (Open Web Text)

Max Activating Token Index: 476

Text #16

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4670. Min Act: -0.1700

Data Index: 1591724 (Open Web Text)

Max Activating Token Index: 143

Text #17

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4618. Min Act: -0.1700

Data Index: 4696267 (Open Web Text)

Max Activating Token Index: 128

Text #18

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4515. Min Act: -0.1700

Data Index: 4526861 (Open Web Text)

Max Activating Token Index: 868

Text #19

Max Range: 4.6021. Min Range: -4.6021

Max Act: 4.4611. Min Act: -0.1700

Data Index: 3820814 (Open Web Text)

Max Activating Token Index: 576