Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2353 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.3806. Min Act: -0.1700

Data Index: 6355333 (Open Web Text)

Max Activating Token Index: 79

Text #1

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.3607. Min Act: -0.1700

Data Index: 3680972 (Open Web Text)

Max Activating Token Index: 606

Text #2

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.3362. Min Act: -0.1700

Data Index: 7744956 (Open Web Text)

Max Activating Token Index: 111

Text #3

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.3290. Min Act: -0.1700

Data Index: 6472852 (Open Web Text)

Max Activating Token Index: 37

Text #4

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.3380. Min Act: -0.1700

Data Index: 2587662 (Open Web Text)

Max Activating Token Index: 628

Text #5

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2745. Min Act: -0.1700

Data Index: 4472675 (Open Web Text)

Max Activating Token Index: 781

Text #6

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2846. Min Act: -0.1700

Data Index: 3748963 (Open Web Text)

Max Activating Token Index: 285

Text #7

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2713. Min Act: -0.1700

Data Index: 8167600 (Open Web Text)

Max Activating Token Index: 188

Text #8

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2745. Min Act: -0.1700

Data Index: 7165496 (Open Web Text)

Max Activating Token Index: 515

Text #9

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2420. Min Act: -0.1700

Data Index: 6278574 (Open Web Text)

Max Activating Token Index: 216

Text #10

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2681. Min Act: -0.1700

Data Index: 2309752 (Open Web Text)

Max Activating Token Index: 95

Text #11

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2493. Min Act: -0.1700

Data Index: 8036473 (Open Web Text)

Max Activating Token Index: 770

Text #12

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2222. Min Act: -0.1700

Data Index: 5004896 (Open Web Text)

Max Activating Token Index: 56

Text #13

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2132. Min Act: -0.1700

Data Index: 6638987 (Open Web Text)

Max Activating Token Index: 55

Text #14

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2115. Min Act: -0.1700

Data Index: 2551857 (Open Web Text)

Max Activating Token Index: 313

Text #15

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.1962. Min Act: -0.1700

Data Index: 3260924 (Open Web Text)

Max Activating Token Index: 450

Text #16

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.1978. Min Act: -0.1700

Data Index: 3717521 (Open Web Text)

Max Activating Token Index: 758

Text #17

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2078. Min Act: -0.1700

Data Index: 1995890 (Open Web Text)

Max Activating Token Index: 85

Text #18

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2070. Min Act: -0.1700

Data Index: 5712354 (Open Web Text)

Max Activating Token Index: 43

Text #19

Max Range: 3.3806. Min Range: -3.3806

Max Act: 3.2094. Min Act: -0.1700

Data Index: 409352 (Open Web Text)

Max Activating Token Index: 30