Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2572 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.3727. Min Act: -0.1700

Data Index: 6008783 (Open Web Text)

Max Activating Token Index: 396

Text #1

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.3741. Min Act: -0.1700

Data Index: 1045768 (Open Web Text)

Max Activating Token Index: 545

Text #2

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2537. Min Act: -0.1700

Data Index: 409466 (Open Web Text)

Max Activating Token Index: 448

Text #3

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2145. Min Act: -0.1700

Data Index: 8198840 (Open Web Text)

Max Activating Token Index: 216

Text #4

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2295. Min Act: -0.1700

Data Index: 8386115 (Open Web Text)

Max Activating Token Index: 592

Text #5

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2131. Min Act: -0.1700

Data Index: 7252150 (Open Web Text)

Max Activating Token Index: 492

Text #6

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1966. Min Act: -0.1700

Data Index: 337997 (Open Web Text)

Max Activating Token Index: 388

Text #7

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1794. Min Act: -0.1700

Data Index: 6353490 (Open Web Text)

Max Activating Token Index: 549

Text #8

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1894. Min Act: -0.1700

Data Index: 3978803 (Open Web Text)

Max Activating Token Index: 903

Text #9

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2035. Min Act: -0.1700

Data Index: 7103457 (Open Web Text)

Max Activating Token Index: 402

Text #10

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.2131. Min Act: -0.1700

Data Index: 650765 (Open Web Text)

Max Activating Token Index: 463

Text #11

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1761. Min Act: -0.1700

Data Index: 657214 (Open Web Text)

Max Activating Token Index: 773

Text #12

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1658. Min Act: -0.1700

Data Index: 2285948 (Open Web Text)

Max Activating Token Index: 489

Text #13

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1650. Min Act: -0.1700

Data Index: 4563002 (Open Web Text)

Max Activating Token Index: 871

Text #14

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1641. Min Act: -0.1700

Data Index: 3391853 (Open Web Text)

Max Activating Token Index: 627

Text #15

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1596. Min Act: -0.1700

Data Index: 211277 (Open Web Text)

Max Activating Token Index: 801

Text #16

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1509. Min Act: -0.1700

Data Index: 101987 (Open Web Text)

Max Activating Token Index: 775

Text #17

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1745. Min Act: -0.1700

Data Index: 7187178 (Open Web Text)

Max Activating Token Index: 869

Text #18

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1778. Min Act: -0.1700

Data Index: 3165994 (Open Web Text)

Max Activating Token Index: 923

Text #19

Max Range: 4.3741. Min Range: -4.3741

Max Act: 4.1660. Min Act: -0.1700

Data Index: 976916 (Open Web Text)

Max Activating Token Index: 344