< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1587 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9558. Min Act: -0.1700

Data Index: 7039068 (Open Web Text)

Max Activating Token Index: 174

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9610. Min Act: -0.1700

Data Index: 3831753 (Open Web Text)

Max Activating Token Index: 71

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9507. Min Act: -0.1700

Data Index: 5353368 (Open Web Text)

Max Activating Token Index: 388

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9369. Min Act: -0.1700

Data Index: 5168232 (Open Web Text)

Max Activating Token Index: 454

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9241. Min Act: -0.1700

Data Index: 2867503 (Open Web Text)

Max Activating Token Index: 268

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9090. Min Act: -0.1700

Data Index: 8292598 (Open Web Text)

Max Activating Token Index: 547

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9098. Min Act: -0.1700

Data Index: 2923149 (Open Web Text)

Max Activating Token Index: 198

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8845. Min Act: -0.1700

Data Index: 329489 (Open Web Text)

Max Activating Token Index: 567

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9001. Min Act: -0.1700

Data Index: 8050616 (Open Web Text)

Max Activating Token Index: 315

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.9039. Min Act: -0.1700

Data Index: 3693646 (Open Web Text)

Max Activating Token Index: 200

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8747. Min Act: -0.1700

Data Index: 2063560 (Open Web Text)

Max Activating Token Index: 347

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8736. Min Act: -0.1700

Data Index: 3066188 (Open Web Text)

Max Activating Token Index: 479

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8700. Min Act: -0.1700

Data Index: 8719892 (Open Web Text)

Max Activating Token Index: 164

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8633. Min Act: -0.1700

Data Index: 5950086 (Open Web Text)

Max Activating Token Index: 423

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8302. Min Act: -0.1700

Data Index: 1299840 (Open Web Text)

Max Activating Token Index: 250

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8589. Min Act: -0.1700

Data Index: 4018583 (Open Web Text)

Max Activating Token Index: 706

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8547. Min Act: -0.1700

Data Index: 1864303 (Open Web Text)

Max Activating Token Index: 361

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8376. Min Act: -0.1700

Data Index: 2275366 (Open Web Text)

Max Activating Token Index: 246

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8388. Min Act: -0.1700

Data Index: 5223192 (Open Web Text)

Max Activating Token Index: 990

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.9610. Min Range: -2.9610

Max Act: 2.8442. Min Act: -0.1700

Data Index: 790628 (Open Web Text)

Max Activating Token Index: 552

Click toggle to see full text

Truncated

Full Text #19