Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 937 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.8353. Min Act: -0.1699

Data Index: 2561713 (Open Web Text)

Max Activating Token Index: 155

Text #1

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7940. Min Act: -0.1700

Data Index: 5660538 (Open Web Text)

Max Activating Token Index: 457

Text #2

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7910. Min Act: -0.1699

Data Index: 3567008 (Open Web Text)

Max Activating Token Index: 524

Text #3

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7733. Min Act: -0.1700

Data Index: 4272975 (Open Web Text)

Max Activating Token Index: 605

Text #4

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7667. Min Act: -0.1697

Data Index: 6203296 (Open Web Text)

Max Activating Token Index: 204

Text #5

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7619. Min Act: -0.1700

Data Index: 5634526 (Open Web Text)

Max Activating Token Index: 774

Text #6

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7640. Min Act: -0.1700

Data Index: 8303212 (Open Web Text)

Max Activating Token Index: 382

Text #7

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7424. Min Act: -0.1699

Data Index: 3188555 (Open Web Text)

Max Activating Token Index: 81

Text #8

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7214. Min Act: -0.1700

Data Index: 8710698 (Open Web Text)

Max Activating Token Index: 616

Text #9

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7471. Min Act: -0.1700

Data Index: 3716495 (Open Web Text)

Max Activating Token Index: 723

Text #10

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7263. Min Act: -0.1700

Data Index: 2457324 (Open Web Text)

Max Activating Token Index: 879

Text #11

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7075. Min Act: -0.1700

Data Index: 5164394 (Open Web Text)

Max Activating Token Index: 587

Text #12

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7106. Min Act: -0.1700

Data Index: 3681373 (Open Web Text)

Max Activating Token Index: 814

Text #13

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7066. Min Act: -0.1700

Data Index: 8138789 (Open Web Text)

Max Activating Token Index: 1016

Text #14

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7110. Min Act: -0.1699

Data Index: 8389726 (Open Web Text)

Max Activating Token Index: 622

Text #15

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7106. Min Act: -0.1700

Data Index: 6916814 (Open Web Text)

Max Activating Token Index: 416

Text #16

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7213. Min Act: -0.1700

Data Index: 5278641 (Open Web Text)

Max Activating Token Index: 107

Text #17

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7116. Min Act: -0.1700

Data Index: 657034 (Open Web Text)

Max Activating Token Index: 770

Text #18

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7214. Min Act: -0.1700

Data Index: 2893464 (Open Web Text)

Max Activating Token Index: 702

Text #19

Max Range: 1.8353. Min Range: -1.8353

Max Act: 1.7218. Min Act: -0.1700

Data Index: 3275756 (Open Web Text)

Max Activating Token Index: 454