Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1158 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8507. Min Act: -0.1486

Data Index: 458868 (Open Web Text)

Max Activating Token Index: 318

Text #1

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8494. Min Act: -0.1308

Data Index: 4075396 (Open Web Text)

Max Activating Token Index: 690

Text #2

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8395. Min Act: -0.1357

Data Index: 447418 (Open Web Text)

Max Activating Token Index: 813

Text #3

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8396. Min Act: -0.1177

Data Index: 7541568 (Open Web Text)

Max Activating Token Index: 732

Text #4

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8282. Min Act: -0.1293

Data Index: 4399121 (Open Web Text)

Max Activating Token Index: 777

Text #5

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8235. Min Act: -0.1035

Data Index: 5942324 (Open Web Text)

Max Activating Token Index: 562

Text #6

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8183. Min Act: -0.1175

Data Index: 2545581 (Open Web Text)

Max Activating Token Index: 314

Text #7

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8213. Min Act: -0.1424

Data Index: 3698460 (Open Web Text)

Max Activating Token Index: 552

Text #8

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8101. Min Act: -0.0908

Data Index: 4997912 (Open Web Text)

Max Activating Token Index: 676

Text #9

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8107. Min Act: -0.1608

Data Index: 1184330 (Open Web Text)

Max Activating Token Index: 333

Text #10

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8094. Min Act: -0.0874

Data Index: 3567319 (Open Web Text)

Max Activating Token Index: 859

Text #11

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8139. Min Act: -0.1117

Data Index: 4868172 (Open Web Text)

Max Activating Token Index: 764

Text #12

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7970. Min Act: -0.1435

Data Index: 8023623 (Open Web Text)

Max Activating Token Index: 694

Text #13

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8033. Min Act: -0.1505

Data Index: 6964241 (Open Web Text)

Max Activating Token Index: 712

Text #14

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.8053. Min Act: -0.0827

Data Index: 6779447 (Open Web Text)

Max Activating Token Index: 320

Text #15

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7922. Min Act: -0.1272

Data Index: 58688 (Open Web Text)

Max Activating Token Index: 878

Text #16

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7983. Min Act: -0.1238

Data Index: 906070 (Open Web Text)

Max Activating Token Index: 569

Text #17

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7907. Min Act: -0.1468

Data Index: 5232988 (Open Web Text)

Max Activating Token Index: 800

Text #18

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7908. Min Act: -0.1417

Data Index: 617591 (Open Web Text)

Max Activating Token Index: 219

Text #19

Max Range: 0.8507. Min Range: -0.8507

Max Act: 0.7901. Min Act: -0.1069

Data Index: 5909516 (Open Web Text)

Max Activating Token Index: 707