< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2067 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.5544. Min Act: -0.1700

Data Index: 2298430 (Open Web Text)

Max Activating Token Index: 454

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3587. Min Act: -0.1700

Data Index: 5765193 (Open Web Text)

Max Activating Token Index: 23

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3279. Min Act: -0.1700

Data Index: 7977057 (Open Web Text)

Max Activating Token Index: 878

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3269. Min Act: -0.1700

Data Index: 3768056 (Open Web Text)

Max Activating Token Index: 124

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3323. Min Act: -0.1700

Data Index: 5446272 (Open Web Text)

Max Activating Token Index: 543

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3202. Min Act: -0.1700

Data Index: 7354955 (Open Web Text)

Max Activating Token Index: 360

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3303. Min Act: -0.1700

Data Index: 8460815 (Open Web Text)

Max Activating Token Index: 864

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3171. Min Act: -0.1700

Data Index: 8183436 (Open Web Text)

Max Activating Token Index: 732

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3185. Min Act: -0.1700

Data Index: 7717444 (Open Web Text)

Max Activating Token Index: 283

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3077. Min Act: -0.1700

Data Index: 70260 (Open Web Text)

Max Activating Token Index: 798

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3150. Min Act: -0.1700

Data Index: 7929202 (Open Web Text)

Max Activating Token Index: 753

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2847. Min Act: -0.1700

Data Index: 2672552 (Open Web Text)

Max Activating Token Index: 977

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2832. Min Act: -0.1700

Data Index: 8117544 (Open Web Text)

Max Activating Token Index: 153

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2812. Min Act: -0.1700

Data Index: 6058106 (Open Web Text)

Max Activating Token Index: 924

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.3092. Min Act: -0.1700

Data Index: 7752472 (Open Web Text)

Max Activating Token Index: 464

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2866. Min Act: -0.1700

Data Index: 7752471 (Open Web Text)

Max Activating Token Index: 724

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2976. Min Act: -0.1700

Data Index: 8243376 (Open Web Text)

Max Activating Token Index: 255

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2731. Min Act: -0.1700

Data Index: 8741018 (Open Web Text)

Max Activating Token Index: 430

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2683. Min Act: -0.1700

Data Index: 3971390 (Open Web Text)

Max Activating Token Index: 440

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.5544. Min Range: -2.5544

Max Act: 2.2373. Min Act: -0.1700

Data Index: 3696431 (Open Web Text)

Max Activating Token Index: 242

Click toggle to see full text

Truncated

Full Text #19