Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 127 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.6409. Min Act: -0.1700

Data Index: 7715658 (Open Web Text)

Max Activating Token Index: 975

Text #1

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.4887. Min Act: -0.1700

Data Index: 7715659 (Open Web Text)

Max Activating Token Index: 410

Text #2

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.4377. Min Act: -0.1700

Data Index: 2230290 (Open Web Text)

Max Activating Token Index: 613

Text #3

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.4275. Min Act: -0.1700

Data Index: 5202713 (Open Web Text)

Max Activating Token Index: 259

Text #4

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.3717. Min Act: -0.1700

Data Index: 646017 (Open Web Text)

Max Activating Token Index: 572

Text #5

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.2457. Min Act: -0.1700

Data Index: 2102133 (Open Web Text)

Max Activating Token Index: 715

Text #6

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.2407. Min Act: -0.1699

Data Index: 1530922 (Open Web Text)

Max Activating Token Index: 183

Text #7

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.2167. Min Act: -0.1700

Data Index: 5202714 (Open Web Text)

Max Activating Token Index: 638

Text #8

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.2033. Min Act: -0.1693

Data Index: 5202715 (Open Web Text)

Max Activating Token Index: 264

Text #9

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1859. Min Act: -0.1700

Data Index: 5202712 (Open Web Text)

Max Activating Token Index: 598

Text #10

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1756. Min Act: -0.1695

Data Index: 8454425 (Open Web Text)

Max Activating Token Index: 752

Text #11

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1520. Min Act: -0.1698

Data Index: 2433981 (Open Web Text)

Max Activating Token Index: 54

Text #12

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1032. Min Act: -0.1700

Data Index: 5668098 (Open Web Text)

Max Activating Token Index: 419

Text #13

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1359. Min Act: -0.1700

Data Index: 646018 (Open Web Text)

Max Activating Token Index: 215

Text #14

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.1013. Min Act: -0.1700

Data Index: 927778 (Open Web Text)

Max Activating Token Index: 620

Text #15

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.0558. Min Act: -0.1700

Data Index: 804748 (Open Web Text)

Max Activating Token Index: 429

Text #16

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.0586. Min Act: -0.1699

Data Index: 2230291 (Open Web Text)

Max Activating Token Index: 98

Text #17

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.0856. Min Act: -0.1700

Data Index: 5870380 (Open Web Text)

Max Activating Token Index: 459

Text #18

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.0448. Min Act: -0.1700

Data Index: 4677854 (Open Web Text)

Max Activating Token Index: 979

Text #19

Max Range: 4.6409. Min Range: -4.6409

Max Act: 4.0598. Min Act: -0.1700

Data Index: 927768 (Open Web Text)

Max Activating Token Index: 878