< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 705 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.2693. Min Act: -0.1700

Data Index: 2961859 (Open Web Text)

Max Activating Token Index: 124

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.2258. Min Act: -0.1700

Data Index: 4019900 (Open Web Text)

Max Activating Token Index: 490

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.2204. Min Act: -0.1700

Data Index: 7310082 (Open Web Text)

Max Activating Token Index: 382

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1947. Min Act: -0.1700

Data Index: 6489797 (Open Web Text)

Max Activating Token Index: 411

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1793. Min Act: -0.1700

Data Index: 1640912 (Open Web Text)

Max Activating Token Index: 943

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1849. Min Act: -0.1700

Data Index: 7692224 (Open Web Text)

Max Activating Token Index: 263

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1558. Min Act: -0.1700

Data Index: 3758745 (Open Web Text)

Max Activating Token Index: 236

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1649. Min Act: -0.1700

Data Index: 1287888 (Open Web Text)

Max Activating Token Index: 91

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1596. Min Act: -0.1700

Data Index: 5992204 (Open Web Text)

Max Activating Token Index: 234

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1348. Min Act: -0.1700

Data Index: 5471505 (Open Web Text)

Max Activating Token Index: 206

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1401. Min Act: -0.1700

Data Index: 2055008 (Open Web Text)

Max Activating Token Index: 332

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1323. Min Act: -0.1700

Data Index: 7427496 (Open Web Text)

Max Activating Token Index: 419

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1284. Min Act: -0.1700

Data Index: 5013541 (Open Web Text)

Max Activating Token Index: 850

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1401. Min Act: -0.1700

Data Index: 8062275 (Open Web Text)

Max Activating Token Index: 123

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1206. Min Act: -0.1700

Data Index: 7352559 (Open Web Text)

Max Activating Token Index: 312

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1060. Min Act: -0.1700

Data Index: 4341877 (Open Web Text)

Max Activating Token Index: 144

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1127. Min Act: -0.1700

Data Index: 3792546 (Open Web Text)

Max Activating Token Index: 419

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.0899. Min Act: -0.1700

Data Index: 5806882 (Open Web Text)

Max Activating Token Index: 276

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1313. Min Act: -0.1700

Data Index: 6506656 (Open Web Text)

Max Activating Token Index: 910

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.2693. Min Range: -2.2693

Max Act: 2.1082. Min Act: -0.1700

Data Index: 8166657 (Open Web Text)

Max Activating Token Index: 34

Click toggle to see full text

Truncated

Full Text #19