< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2013 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.9864. Min Act: -0.1700

Data Index: 2432736 (Open Web Text)

Max Activating Token Index: 134

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.9727. Min Act: -0.1700

Data Index: 1873996 (Open Web Text)

Max Activating Token Index: 398

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.9124. Min Act: -0.1700

Data Index: 6354658 (Open Web Text)

Max Activating Token Index: 143

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8747. Min Act: -0.1700

Data Index: 2658069 (Open Web Text)

Max Activating Token Index: 41

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8478. Min Act: -0.1700

Data Index: 1301275 (Open Web Text)

Max Activating Token Index: 614

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8690. Min Act: -0.1700

Data Index: 7602655 (Open Web Text)

Max Activating Token Index: 818

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8679. Min Act: -0.1700

Data Index: 2372650 (Open Web Text)

Max Activating Token Index: 14

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8862. Min Act: -0.1700

Data Index: 5592046 (Open Web Text)

Max Activating Token Index: 973

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8610. Min Act: -0.1700

Data Index: 8336920 (Open Web Text)

Max Activating Token Index: 501

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8367. Min Act: -0.1700

Data Index: 5087391 (Open Web Text)

Max Activating Token Index: 133

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8648. Min Act: -0.1700

Data Index: 7264273 (Open Web Text)

Max Activating Token Index: 685

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8405. Min Act: -0.1700

Data Index: 8749498 (Open Web Text)

Max Activating Token Index: 510

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8118. Min Act: -0.1700

Data Index: 3718383 (Open Web Text)

Max Activating Token Index: 545

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8111. Min Act: -0.1700

Data Index: 4496866 (Open Web Text)

Max Activating Token Index: 317

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.7685. Min Act: -0.1700

Data Index: 2089810 (Open Web Text)

Max Activating Token Index: 416

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8181. Min Act: -0.1700

Data Index: 1332493 (Open Web Text)

Max Activating Token Index: 855

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8033. Min Act: -0.1700

Data Index: 2476374 (Open Web Text)

Max Activating Token Index: 1002

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8089. Min Act: -0.1700

Data Index: 1322194 (Open Web Text)

Max Activating Token Index: 885

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8204. Min Act: -0.1700

Data Index: 89147 (Open Web Text)

Max Activating Token Index: 796

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.9864. Min Range: -2.9864

Max Act: 2.8022. Min Act: -0.1700

Data Index: 417676 (Open Web Text)

Max Activating Token Index: 590

Click toggle to see full text

Truncated

Full Text #19