Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1212 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5855. Min Act: -0.1629

Data Index: 6058620 (Open Web Text)

Max Activating Token Index: 117

Text #1

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5769. Min Act: -0.1537

Data Index: 1015700 (Open Web Text)

Max Activating Token Index: 394

Text #2

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5715. Min Act: -0.1451

Data Index: 5073645 (Open Web Text)

Max Activating Token Index: 399

Text #3

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5674. Min Act: -0.1700

Data Index: 5139761 (Open Web Text)

Max Activating Token Index: 390

Text #4

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5623. Min Act: -0.1681

Data Index: 4329301 (Open Web Text)

Max Activating Token Index: 438

Text #5

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5537. Min Act: -0.1615

Data Index: 4278733 (Open Web Text)

Max Activating Token Index: 673

Text #6

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5313. Min Act: -0.1655

Data Index: 275830 (Open Web Text)

Max Activating Token Index: 496

Text #7

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5434. Min Act: -0.1691

Data Index: 873175 (Open Web Text)

Max Activating Token Index: 270

Text #8

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5391. Min Act: -0.1562

Data Index: 7120931 (Open Web Text)

Max Activating Token Index: 395

Text #9

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5376. Min Act: -0.1692

Data Index: 8303783 (Open Web Text)

Max Activating Token Index: 395

Text #10

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5309. Min Act: -0.1432

Data Index: 5246387 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5269. Min Act: -0.1423

Data Index: 2735504 (Open Web Text)

Max Activating Token Index: 271

Text #12

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5407. Min Act: -0.1411

Data Index: 4396394 (Open Web Text)

Max Activating Token Index: 599

Text #13

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5260. Min Act: -0.1555

Data Index: 7334460 (Open Web Text)

Max Activating Token Index: 715

Text #14

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5228. Min Act: -0.1642

Data Index: 4049667 (Open Web Text)

Max Activating Token Index: 76

Text #15

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5239. Min Act: -0.1625

Data Index: 4275680 (Open Web Text)

Max Activating Token Index: 249

Text #16

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5255. Min Act: -0.1552

Data Index: 4717388 (Open Web Text)

Max Activating Token Index: 690

Text #17

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5214. Min Act: -0.1608

Data Index: 5246822 (Open Web Text)

Max Activating Token Index: 767

Text #18

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5249. Min Act: -0.1326

Data Index: 7935467 (Open Web Text)

Max Activating Token Index: 271

Text #19

Max Range: 1.5855. Min Range: -1.5855

Max Act: 1.5235. Min Act: -0.1682

Data Index: 5491302 (Open Web Text)

Max Activating Token Index: 207