Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1874 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9159. Min Act: -0.1317

Data Index: 4187269 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9814. Min Act: -0.1395

Data Index: 3345396 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9694. Min Act: -0.1698

Data Index: 3576182 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9530. Min Act: -0.1108

Data Index: 3956799 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9293. Min Act: -0.1585

Data Index: 2242558 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9459. Min Act: -0.1337

Data Index: 3438880 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8748. Min Act: -0.1571

Data Index: 3822148 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8758. Min Act: -0.1644

Data Index: 8222089 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8943. Min Act: -0.1205

Data Index: 3690462 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9089. Min Act: -0.1689

Data Index: 58720 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9118. Min Act: -0.1674

Data Index: 5973126 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.9044. Min Act: -0.1184

Data Index: 130949 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8881. Min Act: -0.1615

Data Index: 2215934 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8894. Min Act: -0.1312

Data Index: 5182162 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8795. Min Act: -0.1433

Data Index: 3607919 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8691. Min Act: -0.1617

Data Index: 4231622 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8701. Min Act: -0.1253

Data Index: 756714 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8592. Min Act: -0.0989

Data Index: 1214789 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8870. Min Act: -0.1700

Data Index: 121704 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 3.9814. Min Range: -3.9814

Max Act: 3.8018. Min Act: -0.1380

Data Index: 1267166 (Open Web Text)

Max Activating Token Index: 1023