< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 116 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.9066. Min Act: -0.1700

Data Index: 1047052 (Open Web Text)

Max Activating Token Index: 843

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.8652. Min Act: -0.1700

Data Index: 6950321 (Open Web Text)

Max Activating Token Index: 551

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.8585. Min Act: -0.1700

Data Index: 5636577 (Open Web Text)

Max Activating Token Index: 273

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.8496. Min Act: -0.1700

Data Index: 4103449 (Open Web Text)

Max Activating Token Index: 613

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.8219. Min Act: -0.1700

Data Index: 1018552 (Open Web Text)

Max Activating Token Index: 41

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7625. Min Act: -0.1700

Data Index: 2195194 (Open Web Text)

Max Activating Token Index: 518

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7529. Min Act: -0.1700

Data Index: 2195200 (Open Web Text)

Max Activating Token Index: 348

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7643. Min Act: -0.1700

Data Index: 8624051 (Open Web Text)

Max Activating Token Index: 571

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7104. Min Act: -0.1700

Data Index: 3759993 (Open Web Text)

Max Activating Token Index: 181

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7307. Min Act: -0.1700

Data Index: 5656702 (Open Web Text)

Max Activating Token Index: 194

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7178. Min Act: -0.1695

Data Index: 5774829 (Open Web Text)

Max Activating Token Index: 243

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7175. Min Act: -0.1700

Data Index: 4117889 (Open Web Text)

Max Activating Token Index: 111

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7251. Min Act: -0.1700

Data Index: 1625826 (Open Web Text)

Max Activating Token Index: 282

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7174. Min Act: -0.1700

Data Index: 3230329 (Open Web Text)

Max Activating Token Index: 55

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7223. Min Act: -0.1700

Data Index: 1298328 (Open Web Text)

Max Activating Token Index: 632

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.6917. Min Act: -0.1700

Data Index: 2992473 (Open Web Text)

Max Activating Token Index: 82

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.7030. Min Act: -0.1700

Data Index: 7274316 (Open Web Text)

Max Activating Token Index: 158

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.6900. Min Act: -0.1700

Data Index: 1616354 (Open Web Text)

Max Activating Token Index: 71

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.6918. Min Act: -0.1700

Data Index: 4001875 (Open Web Text)

Max Activating Token Index: 582

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.9066. Min Range: -2.9066

Max Act: 2.6940. Min Act: -0.1700

Data Index: 5099464 (Open Web Text)

Max Activating Token Index: 146

Click toggle to see full text

Truncated

Full Text #19