Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1057 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.0764. Min Range: -7.0764

Max Act: 7.0757. Min Act: -0.1700

Data Index: 4291721 (Open Web Text)

Max Activating Token Index: 64

Text #1

Max Range: 7.0764. Min Range: -7.0764

Max Act: 7.0764. Min Act: -0.1700

Data Index: 4221599 (Open Web Text)

Max Activating Token Index: 87

Text #2

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.9733. Min Act: -0.1700

Data Index: 2966626 (Open Web Text)

Max Activating Token Index: 749

Text #3

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8880. Min Act: -0.1700

Data Index: 8493902 (Open Web Text)

Max Activating Token Index: 242

Text #4

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8637. Min Act: -0.1700

Data Index: 4613338 (Open Web Text)

Max Activating Token Index: 71

Text #5

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8729. Min Act: -0.1700

Data Index: 3585426 (Open Web Text)

Max Activating Token Index: 683

Text #6

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8531. Min Act: -0.1700

Data Index: 2574308 (Open Web Text)

Max Activating Token Index: 66

Text #7

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8605. Min Act: -0.1700

Data Index: 2806637 (Open Web Text)

Max Activating Token Index: 100

Text #8

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8073. Min Act: -0.1700

Data Index: 5080106 (Open Web Text)

Max Activating Token Index: 105

Text #9

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8029. Min Act: -0.1700

Data Index: 7405792 (Open Web Text)

Max Activating Token Index: 87

Text #10

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7858. Min Act: -0.1700

Data Index: 5465442 (Open Web Text)

Max Activating Token Index: 136

Text #11

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8785. Min Act: -0.1700

Data Index: 1316602 (Open Web Text)

Max Activating Token Index: 57

Text #12

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7566. Min Act: -0.1700

Data Index: 1488185 (Open Web Text)

Max Activating Token Index: 181

Text #13

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7785. Min Act: -0.1700

Data Index: 4671698 (Open Web Text)

Max Activating Token Index: 82

Text #14

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.8014. Min Act: -0.1700

Data Index: 3719423 (Open Web Text)

Max Activating Token Index: 120

Text #15

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7970. Min Act: -0.1700

Data Index: 3798337 (Open Web Text)

Max Activating Token Index: 67

Text #16

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7680. Min Act: -0.1700

Data Index: 8164212 (Open Web Text)

Max Activating Token Index: 127

Text #17

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7195. Min Act: -0.1700

Data Index: 6850650 (Open Web Text)

Max Activating Token Index: 85

Text #18

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7716. Min Act: -0.1700

Data Index: 1000491 (Open Web Text)

Max Activating Token Index: 117

Text #19

Max Range: 7.0764. Min Range: -7.0764

Max Act: 6.7648. Min Act: -0.1700

Data Index: 3613468 (Open Web Text)

Max Activating Token Index: 130