< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 582 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.9907. Min Act: -0.1700

Data Index: 7923384 (Open Web Text)

Max Activating Token Index: 438

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.9335. Min Act: -0.1700

Data Index: 5478609 (Open Web Text)

Max Activating Token Index: 599

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.9019. Min Act: -0.1700

Data Index: 810559 (Open Web Text)

Max Activating Token Index: 490

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8869. Min Act: -0.1700

Data Index: 262111 (Open Web Text)

Max Activating Token Index: 812

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8487. Min Act: -0.1700

Data Index: 8602097 (Open Web Text)

Max Activating Token Index: 484

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8591. Min Act: -0.1700

Data Index: 1929171 (Open Web Text)

Max Activating Token Index: 569

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8541. Min Act: -0.1700

Data Index: 2948568 (Open Web Text)

Max Activating Token Index: 92

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8205. Min Act: -0.1700

Data Index: 70491 (Open Web Text)

Max Activating Token Index: 306

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8235. Min Act: -0.1700

Data Index: 2758565 (Open Web Text)

Max Activating Token Index: 300

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.8223. Min Act: -0.1700

Data Index: 2231398 (Open Web Text)

Max Activating Token Index: 20

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7933. Min Act: -0.1700

Data Index: 7605845 (Open Web Text)

Max Activating Token Index: 960

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7969. Min Act: -0.1700

Data Index: 4720773 (Open Web Text)

Max Activating Token Index: 765

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7833. Min Act: -0.1700

Data Index: 2368420 (Open Web Text)

Max Activating Token Index: 97

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7765. Min Act: -0.1700

Data Index: 6691891 (Open Web Text)

Max Activating Token Index: 591

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7682. Min Act: -0.1700

Data Index: 734291 (Open Web Text)

Max Activating Token Index: 631

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7621. Min Act: -0.1700

Data Index: 8273099 (Open Web Text)

Max Activating Token Index: 208

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7766. Min Act: -0.1700

Data Index: 6787006 (Open Web Text)

Max Activating Token Index: 748

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7562. Min Act: -0.1700

Data Index: 3463224 (Open Web Text)

Max Activating Token Index: 230

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7334. Min Act: -0.1700

Data Index: 5731952 (Open Web Text)

Max Activating Token Index: 529

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.9907. Min Range: -2.9907

Max Act: 2.7254. Min Act: -0.1700

Data Index: 1904481 (Open Web Text)

Max Activating Token Index: 1016

Click toggle to see full text

Truncated

Full Text #19