Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 658 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9804. Min Act: -0.1700

Data Index: 2760078 (Open Web Text)

Max Activating Token Index: 855

Text #1

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9884. Min Act: -0.1700

Data Index: 5523101 (Open Web Text)

Max Activating Token Index: 791

Text #2

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9250. Min Act: -0.1700

Data Index: 5673788 (Open Web Text)

Max Activating Token Index: 781

Text #3

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9143. Min Act: -0.1700

Data Index: 2734221 (Open Web Text)

Max Activating Token Index: 230

Text #4

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9313. Min Act: -0.1700

Data Index: 32172 (Open Web Text)

Max Activating Token Index: 593

Text #5

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9413. Min Act: -0.1700

Data Index: 6662984 (Open Web Text)

Max Activating Token Index: 171

Text #6

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.9162. Min Act: -0.1700

Data Index: 6685446 (Open Web Text)

Max Activating Token Index: 576

Text #7

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8883. Min Act: -0.1700

Data Index: 4032986 (Open Web Text)

Max Activating Token Index: 130

Text #8

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8757. Min Act: -0.1700

Data Index: 7678182 (Open Web Text)

Max Activating Token Index: 47

Text #9

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8686. Min Act: -0.1700

Data Index: 1599548 (Open Web Text)

Max Activating Token Index: 655

Text #10

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8043. Min Act: -0.1700

Data Index: 3680037 (Open Web Text)

Max Activating Token Index: 122

Text #11

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7894. Min Act: -0.1700

Data Index: 2021998 (Open Web Text)

Max Activating Token Index: 769

Text #12

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8092. Min Act: -0.1700

Data Index: 748542 (Open Web Text)

Max Activating Token Index: 167

Text #13

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8365. Min Act: -0.1700

Data Index: 2733633 (Open Web Text)

Max Activating Token Index: 340

Text #14

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.8289. Min Act: -0.1700

Data Index: 1435652 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7949. Min Act: -0.1700

Data Index: 3908149 (Open Web Text)

Max Activating Token Index: 817

Text #16

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7960. Min Act: -0.1700

Data Index: 1180929 (Open Web Text)

Max Activating Token Index: 521

Text #17

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7673. Min Act: -0.1700

Data Index: 8417176 (Open Web Text)

Max Activating Token Index: 592

Text #18

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7924. Min Act: -0.1700

Data Index: 1969430 (Open Web Text)

Max Activating Token Index: 759

Text #19

Max Range: 5.9884. Min Range: -5.9884

Max Act: 5.7955. Min Act: -0.1700

Data Index: 4305737 (Open Web Text)

Max Activating Token Index: 546