Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1857 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.7458. Min Act: -0.1700

Data Index: 8774840 (Open Web Text)

Max Activating Token Index: 335

Text #1

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.6757. Min Act: -0.1700

Data Index: 1832367 (Open Web Text)

Max Activating Token Index: 251

Text #2

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.6615. Min Act: -0.1700

Data Index: 5493701 (Open Web Text)

Max Activating Token Index: 783

Text #3

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.6007. Min Act: -0.1700

Data Index: 5091898 (Open Web Text)

Max Activating Token Index: 21

Text #4

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5580. Min Act: -0.1700

Data Index: 4450399 (Open Web Text)

Max Activating Token Index: 252

Text #5

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5333. Min Act: -0.1700

Data Index: 1905334 (Open Web Text)

Max Activating Token Index: 265

Text #6

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5167. Min Act: -0.1700

Data Index: 6183023 (Open Web Text)

Max Activating Token Index: 379

Text #7

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5621. Min Act: -0.1700

Data Index: 4062823 (Open Web Text)

Max Activating Token Index: 947

Text #8

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5468. Min Act: -0.1700

Data Index: 557018 (Open Web Text)

Max Activating Token Index: 92

Text #9

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5300. Min Act: -0.1700

Data Index: 2681965 (Open Web Text)

Max Activating Token Index: 911

Text #10

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5130. Min Act: -0.1700

Data Index: 831262 (Open Web Text)

Max Activating Token Index: 269

Text #11

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5148. Min Act: -0.1700

Data Index: 5091897 (Open Web Text)

Max Activating Token Index: 279

Text #12

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5357. Min Act: -0.1700

Data Index: 6458855 (Open Web Text)

Max Activating Token Index: 718

Text #13

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5556. Min Act: -0.1700

Data Index: 1963035 (Open Web Text)

Max Activating Token Index: 352

Text #14

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5210. Min Act: -0.1700

Data Index: 2394002 (Open Web Text)

Max Activating Token Index: 701

Text #15

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5142. Min Act: -0.1700

Data Index: 3930557 (Open Web Text)

Max Activating Token Index: 85

Text #16

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5035. Min Act: -0.1700

Data Index: 2320788 (Open Web Text)

Max Activating Token Index: 759

Text #17

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5459. Min Act: -0.1700

Data Index: 6411637 (Open Web Text)

Max Activating Token Index: 986

Text #18

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.5079. Min Act: -0.1700

Data Index: 1277453 (Open Web Text)

Max Activating Token Index: 512

Text #19

Max Range: 3.7458. Min Range: -3.7458

Max Act: 3.4792. Min Act: -0.1700

Data Index: 6904873 (Open Web Text)

Max Activating Token Index: 104