Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 962 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6945. Min Act: -0.1700

Data Index: 5606658 (Open Web Text)

Max Activating Token Index: 39

Text #1

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6915. Min Act: -0.1700

Data Index: 1136170 (Open Web Text)

Max Activating Token Index: 457

Text #2

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6907. Min Act: -0.1700

Data Index: 1055874 (Open Web Text)

Max Activating Token Index: 36

Text #3

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6904. Min Act: -0.1700

Data Index: 2873690 (Open Web Text)

Max Activating Token Index: 34

Text #4

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6885. Min Act: -0.1700

Data Index: 4891453 (Open Web Text)

Max Activating Token Index: 38

Text #5

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6846. Min Act: -0.1700

Data Index: 2797166 (Open Web Text)

Max Activating Token Index: 37

Text #6

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6830. Min Act: -0.1700

Data Index: 5983537 (Open Web Text)

Max Activating Token Index: 451

Text #7

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6821. Min Act: -0.1700

Data Index: 4282313 (Open Web Text)

Max Activating Token Index: 43

Text #8

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6705. Min Act: -0.1700

Data Index: 2547686 (Open Web Text)

Max Activating Token Index: 36

Text #9

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6708. Min Act: -0.1700

Data Index: 7344391 (Open Web Text)

Max Activating Token Index: 31

Text #10

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6611. Min Act: -0.1700

Data Index: 50413 (Open Web Text)

Max Activating Token Index: 37

Text #11

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6629. Min Act: -0.1700

Data Index: 8585619 (Open Web Text)

Max Activating Token Index: 457

Text #12

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6561. Min Act: -0.1700

Data Index: 2135500 (Open Web Text)

Max Activating Token Index: 37

Text #13

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6630. Min Act: -0.1700

Data Index: 3287751 (Open Web Text)

Max Activating Token Index: 37

Text #14

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6705. Min Act: -0.1700

Data Index: 5879136 (Open Web Text)

Max Activating Token Index: 457

Text #15

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6517. Min Act: -0.1700

Data Index: 4150273 (Open Web Text)

Max Activating Token Index: 453

Text #16

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6499. Min Act: -0.1700

Data Index: 8265640 (Open Web Text)

Max Activating Token Index: 462

Text #17

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6649. Min Act: -0.1700

Data Index: 2845106 (Open Web Text)

Max Activating Token Index: 445

Text #18

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6649. Min Act: -0.1700

Data Index: 7216 (Open Web Text)

Max Activating Token Index: 461

Text #19

Max Range: 0.6945. Min Range: -0.6945

Max Act: 0.6438. Min Act: -0.1700

Data Index: 2941834 (Open Web Text)

Max Activating Token Index: 34