Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 550 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.5636. Min Act: -0.1700

Data Index: 2092301 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.5653. Min Act: -0.1657

Data Index: 564579 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.5237. Min Act: -0.1556

Data Index: 2836343 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.5142. Min Act: -0.1285

Data Index: 4742175 (Open Web Text)

Max Activating Token Index: 241

Text #4

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4874. Min Act: -0.1649

Data Index: 5811006 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4803. Min Act: -0.1423

Data Index: 8187179 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4718. Min Act: -0.1520

Data Index: 1715986 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4765. Min Act: -0.1549

Data Index: 5860981 (Open Web Text)

Max Activating Token Index: 648

Text #8

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4550. Min Act: -0.1649

Data Index: 5476026 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4624. Min Act: -0.1342

Data Index: 622901 (Open Web Text)

Max Activating Token Index: 408

Text #10

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4617. Min Act: -0.1378

Data Index: 6875555 (Open Web Text)

Max Activating Token Index: 486

Text #11

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4776. Min Act: -0.1520

Data Index: 6168615 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4530. Min Act: -0.1574

Data Index: 5430118 (Open Web Text)

Max Activating Token Index: 85

Text #13

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4434. Min Act: -0.1684

Data Index: 3955486 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4579. Min Act: -0.1509

Data Index: 7062580 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4647. Min Act: -0.1641

Data Index: 7940226 (Open Web Text)

Max Activating Token Index: 212

Text #16

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4639. Min Act: -0.1301

Data Index: 859709 (Open Web Text)

Max Activating Token Index: 582

Text #17

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4606. Min Act: -0.1571

Data Index: 4609072 (Open Web Text)

Max Activating Token Index: 172

Text #18

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4578. Min Act: -0.1697

Data Index: 3644883 (Open Web Text)

Max Activating Token Index: 298

Text #19

Max Range: 1.5653. Min Range: -1.5653

Max Act: 1.4510. Min Act: -0.1534

Data Index: 3816996 (Open Web Text)

Max Activating Token Index: 315