Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1341 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0752. Min Range: -4.0752

Max Act: 4.0752. Min Act: -0.1700

Data Index: 4987060 (Open Web Text)

Max Activating Token Index: 757

Text #1

Max Range: 4.0752. Min Range: -4.0752

Max Act: 4.0089. Min Act: -0.1700

Data Index: 6038202 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 4.0752. Min Range: -4.0752

Max Act: 4.0021. Min Act: -0.1700

Data Index: 4987061 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.9382. Min Act: -0.1700

Data Index: 5943695 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.9594. Min Act: -0.1700

Data Index: 3535841 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.9154. Min Act: -0.1700

Data Index: 3084541 (Open Web Text)

Max Activating Token Index: 8

Text #6

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.9023. Min Act: -0.1700

Data Index: 4878497 (Open Web Text)

Max Activating Token Index: 920

Text #7

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.9221. Min Act: -0.1700

Data Index: 4168296 (Open Web Text)

Max Activating Token Index: 896

Text #8

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8872. Min Act: -0.1700

Data Index: 7082290 (Open Web Text)

Max Activating Token Index: 32

Text #9

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8576. Min Act: -0.1700

Data Index: 1561156 (Open Web Text)

Max Activating Token Index: 989

Text #10

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8811. Min Act: -0.1700

Data Index: 7608389 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8676. Min Act: -0.1700

Data Index: 3276561 (Open Web Text)

Max Activating Token Index: 16

Text #12

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8843. Min Act: -0.1700

Data Index: 1372938 (Open Web Text)

Max Activating Token Index: 886

Text #13

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8585. Min Act: -0.1700

Data Index: 2674060 (Open Web Text)

Max Activating Token Index: 836

Text #14

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8573. Min Act: -0.1700

Data Index: 5955791 (Open Web Text)

Max Activating Token Index: 276

Text #15

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8516. Min Act: -0.1700

Data Index: 3695848 (Open Web Text)

Max Activating Token Index: 947

Text #16

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8584. Min Act: -0.1700

Data Index: 4353966 (Open Web Text)

Max Activating Token Index: 852

Text #17

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8584. Min Act: -0.1700

Data Index: 8036801 (Open Web Text)

Max Activating Token Index: 294

Text #18

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8552. Min Act: -0.1700

Data Index: 5296510 (Open Web Text)

Max Activating Token Index: 589

Text #19

Max Range: 4.0752. Min Range: -4.0752

Max Act: 3.8184. Min Act: -0.1700

Data Index: 182339 (Open Web Text)

Max Activating Token Index: 908