Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1780 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2469. Min Act: -0.1700

Data Index: 2103922 (Open Web Text)

Max Activating Token Index: 532

Text #1

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2261. Min Act: -0.1700

Data Index: 5104120 (Open Web Text)

Max Activating Token Index: 17

Text #2

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1919. Min Act: -0.1700

Data Index: 508240 (Open Web Text)

Max Activating Token Index: 8

Text #3

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2146. Min Act: -0.1700

Data Index: 2263836 (Open Web Text)

Max Activating Token Index: 27

Text #4

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1912. Min Act: -0.1700

Data Index: 922118 (Open Web Text)

Max Activating Token Index: 141

Text #5

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2387. Min Act: -0.1700

Data Index: 7469758 (Open Web Text)

Max Activating Token Index: 381

Text #6

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1764. Min Act: -0.1700

Data Index: 6397192 (Open Web Text)

Max Activating Token Index: 20

Text #7

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1772. Min Act: -0.1700

Data Index: 5352075 (Open Web Text)

Max Activating Token Index: 25

Text #8

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1790. Min Act: -0.1700

Data Index: 1097772 (Open Web Text)

Max Activating Token Index: 788

Text #9

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1515. Min Act: -0.1700

Data Index: 397035 (Open Web Text)

Max Activating Token Index: 478

Text #10

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2058. Min Act: -0.1700

Data Index: 1777495 (Open Web Text)

Max Activating Token Index: 753

Text #11

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.2003. Min Act: -0.1700

Data Index: 4853334 (Open Web Text)

Max Activating Token Index: 746

Text #12

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1198. Min Act: -0.1700

Data Index: 5014178 (Open Web Text)

Max Activating Token Index: 508

Text #13

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1551. Min Act: -0.1700

Data Index: 1258543 (Open Web Text)

Max Activating Token Index: 528

Text #14

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1180. Min Act: -0.1700

Data Index: 1773529 (Open Web Text)

Max Activating Token Index: 760

Text #15

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1467. Min Act: -0.1700

Data Index: 6761198 (Open Web Text)

Max Activating Token Index: 595

Text #16

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1568. Min Act: -0.1700

Data Index: 2255558 (Open Web Text)

Max Activating Token Index: 683

Text #17

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1527. Min Act: -0.1700

Data Index: 7613315 (Open Web Text)

Max Activating Token Index: 505

Text #18

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1410. Min Act: -0.1700

Data Index: 592110 (Open Web Text)

Max Activating Token Index: 781

Text #19

Max Range: 4.2469. Min Range: -4.2469

Max Act: 4.1697. Min Act: -0.1700

Data Index: 3427662 (Open Web Text)

Max Activating Token Index: 747