< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 323 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0807. Min Act: -0.1700

Data Index: 790624 (Open Web Text)

Max Activating Token Index: 32

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0684. Min Act: -0.1700

Data Index: 8420959 (Open Web Text)

Max Activating Token Index: 50

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0391. Min Act: -0.1700

Data Index: 65845 (Open Web Text)

Max Activating Token Index: 955

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0524. Min Act: -0.1700

Data Index: 1285888 (Open Web Text)

Max Activating Token Index: 562

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0591. Min Act: -0.1700

Data Index: 3917026 (Open Web Text)

Max Activating Token Index: 417

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0091. Min Act: -0.1700

Data Index: 5093167 (Open Web Text)

Max Activating Token Index: 754

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9996. Min Act: -0.1700

Data Index: 6128914 (Open Web Text)

Max Activating Token Index: 191

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0223. Min Act: -0.1700

Data Index: 3657438 (Open Web Text)

Max Activating Token Index: 632

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9838. Min Act: -0.1700

Data Index: 7147040 (Open Web Text)

Max Activating Token Index: 464

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9938. Min Act: -0.1700

Data Index: 2711795 (Open Web Text)

Max Activating Token Index: 10

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0039. Min Act: -0.1700

Data Index: 7097475 (Open Web Text)

Max Activating Token Index: 156

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.0807. Min Range: -2.0807

Max Act: 2.0034. Min Act: -0.1700

Data Index: 4054373 (Open Web Text)

Max Activating Token Index: 459

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9868. Min Act: -0.1700

Data Index: 6695482 (Open Web Text)

Max Activating Token Index: 12

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9945. Min Act: -0.1700

Data Index: 7418765 (Open Web Text)

Max Activating Token Index: 606

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9956. Min Act: -0.1700

Data Index: 5990173 (Open Web Text)

Max Activating Token Index: 539

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9863. Min Act: -0.1700

Data Index: 1000297 (Open Web Text)

Max Activating Token Index: 769

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9746. Min Act: -0.1700

Data Index: 3787889 (Open Web Text)

Max Activating Token Index: 457

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9854. Min Act: -0.1700

Data Index: 6815677 (Open Web Text)

Max Activating Token Index: 524

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9789. Min Act: -0.1700

Data Index: 380387 (Open Web Text)

Max Activating Token Index: 1009

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.0807. Min Range: -2.0807

Max Act: 1.9759. Min Act: -0.1700

Data Index: 5555032 (Open Web Text)

Max Activating Token Index: 765

Click toggle to see full text

Truncated

Full Text #19