Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2084 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.9481. Min Act: -0.1700

Data Index: 6886130 (Open Web Text)

Max Activating Token Index: 632

Text #1

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.6955. Min Act: -0.1683

Data Index: 5935500 (Open Web Text)

Max Activating Token Index: 656

Text #2

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.7127. Min Act: -0.1699

Data Index: 8565970 (Open Web Text)

Max Activating Token Index: 792

Text #3

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.7620. Min Act: -0.1243

Data Index: 2217176 (Open Web Text)

Max Activating Token Index: 865

Text #4

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.6107. Min Act: -0.1683

Data Index: 4047828 (Open Web Text)

Max Activating Token Index: 218

Text #5

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.6735. Min Act: -0.1694

Data Index: 2947007 (Open Web Text)

Max Activating Token Index: 390

Text #6

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5764. Min Act: -0.1697

Data Index: 1993337 (Open Web Text)

Max Activating Token Index: 259

Text #7

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5616. Min Act: -0.1698

Data Index: 8591839 (Open Web Text)

Max Activating Token Index: 841

Text #8

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.6027. Min Act: -0.1403

Data Index: 3862423 (Open Web Text)

Max Activating Token Index: 346

Text #9

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.4971. Min Act: -0.1637

Data Index: 7835336 (Open Web Text)

Max Activating Token Index: 758

Text #10

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.4936. Min Act: -0.1700

Data Index: 910258 (Open Web Text)

Max Activating Token Index: 495

Text #11

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5241. Min Act: -0.1699

Data Index: 4045044 (Open Web Text)

Max Activating Token Index: 260

Text #12

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5444. Min Act: -0.1695

Data Index: 6407614 (Open Web Text)

Max Activating Token Index: 519

Text #13

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5363. Min Act: -0.1700

Data Index: 4926845 (Open Web Text)

Max Activating Token Index: 631

Text #14

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5282. Min Act: -0.1700

Data Index: 4184119 (Open Web Text)

Max Activating Token Index: 307

Text #15

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.5023. Min Act: -0.1700

Data Index: 366587 (Open Web Text)

Max Activating Token Index: 87

Text #16

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.3517. Min Act: -0.1693

Data Index: 8089969 (Open Web Text)

Max Activating Token Index: 581

Text #17

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.4758. Min Act: -0.1700

Data Index: 3639927 (Open Web Text)

Max Activating Token Index: 36

Text #18

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.4758. Min Act: -0.1700

Data Index: 7788786 (Open Web Text)

Max Activating Token Index: 392

Text #19

Max Range: 5.9481. Min Range: -5.9481

Max Act: 5.4469. Min Act: -0.1700

Data Index: 5177140 (Open Web Text)

Max Activating Token Index: 935