Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 303 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9604. Min Act: -0.1665

Data Index: 1961240 (Open Web Text)

Max Activating Token Index: 143

Text #1

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9516. Min Act: -0.1700

Data Index: 178575 (Open Web Text)

Max Activating Token Index: 480

Text #2

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9476. Min Act: -0.1700

Data Index: 7162952 (Open Web Text)

Max Activating Token Index: 989

Text #3

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9524. Min Act: -0.1700

Data Index: 1029408 (Open Web Text)

Max Activating Token Index: 490

Text #4

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9386. Min Act: -0.1698

Data Index: 2528456 (Open Web Text)

Max Activating Token Index: 505

Text #5

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9206. Min Act: -0.1698

Data Index: 4891256 (Open Web Text)

Max Activating Token Index: 523

Text #6

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9270. Min Act: -0.1700

Data Index: 4864728 (Open Web Text)

Max Activating Token Index: 380

Text #7

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9224. Min Act: -0.1700

Data Index: 5762652 (Open Web Text)

Max Activating Token Index: 1012

Text #8

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9131. Min Act: -0.1693

Data Index: 5923762 (Open Web Text)

Max Activating Token Index: 71

Text #9

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9055. Min Act: -0.1700

Data Index: 4436980 (Open Web Text)

Max Activating Token Index: 887

Text #10

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9279. Min Act: -0.1700

Data Index: 3188991 (Open Web Text)

Max Activating Token Index: 666

Text #11

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9146. Min Act: -0.1700

Data Index: 7427955 (Open Web Text)

Max Activating Token Index: 573

Text #12

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.9110. Min Act: -0.1699

Data Index: 5157000 (Open Web Text)

Max Activating Token Index: 506

Text #13

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8956. Min Act: -0.1690

Data Index: 1626142 (Open Web Text)

Max Activating Token Index: 81

Text #14

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8981. Min Act: -0.1698

Data Index: 2323921 (Open Web Text)

Max Activating Token Index: 654

Text #15

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8984. Min Act: -0.1683

Data Index: 6825132 (Open Web Text)

Max Activating Token Index: 100

Text #16

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8966. Min Act: -0.1700

Data Index: 4802202 (Open Web Text)

Max Activating Token Index: 520

Text #17

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8986. Min Act: -0.1700

Data Index: 43702 (Open Web Text)

Max Activating Token Index: 68

Text #18

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8934. Min Act: -0.1700

Data Index: 7074833 (Open Web Text)

Max Activating Token Index: 444

Text #19

Max Range: 1.9604. Min Range: -1.9604

Max Act: 1.8868. Min Act: -0.1700

Data Index: 5248850 (Open Web Text)

Max Activating Token Index: 79