Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 417 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.2542. Min Act: -0.1700

Data Index: 3343760 (Open Web Text)

Max Activating Token Index: 107

Text #1

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1648. Min Act: -0.1700

Data Index: 2979727 (Open Web Text)

Max Activating Token Index: 510

Text #2

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1508. Min Act: -0.1700

Data Index: 1688792 (Open Web Text)

Max Activating Token Index: 54

Text #3

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1597. Min Act: -0.1700

Data Index: 3834941 (Open Web Text)

Max Activating Token Index: 511

Text #4

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1196. Min Act: -0.1700

Data Index: 7576620 (Open Web Text)

Max Activating Token Index: 55

Text #5

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1192. Min Act: -0.1700

Data Index: 6039208 (Open Web Text)

Max Activating Token Index: 202

Text #6

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1188. Min Act: -0.1700

Data Index: 608959 (Open Web Text)

Max Activating Token Index: 90

Text #7

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.1443. Min Act: -0.1700

Data Index: 4318385 (Open Web Text)

Max Activating Token Index: 182

Text #8

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0960. Min Act: -0.1700

Data Index: 7120636 (Open Web Text)

Max Activating Token Index: 715

Text #9

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0868. Min Act: -0.1700

Data Index: 4894185 (Open Web Text)

Max Activating Token Index: 379

Text #10

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0966. Min Act: -0.1700

Data Index: 2979734 (Open Web Text)

Max Activating Token Index: 252

Text #11

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0078. Min Act: -0.1700

Data Index: 3743195 (Open Web Text)

Max Activating Token Index: 113

Text #12

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0253. Min Act: -0.1700

Data Index: 2978881 (Open Web Text)

Max Activating Token Index: 68

Text #13

Max Range: 4.2542. Min Range: -4.2542

Max Act: 4.0094. Min Act: -0.1700

Data Index: 3352716 (Open Web Text)

Max Activating Token Index: 123

Text #14

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9899. Min Act: -0.1700

Data Index: 4985397 (Open Web Text)

Max Activating Token Index: 654

Text #15

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9753. Min Act: -0.1700

Data Index: 1763815 (Open Web Text)

Max Activating Token Index: 96

Text #16

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9729. Min Act: -0.1700

Data Index: 5963064 (Open Web Text)

Max Activating Token Index: 608

Text #17

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9612. Min Act: -0.1700

Data Index: 4175053 (Open Web Text)

Max Activating Token Index: 1014

Text #18

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9738. Min Act: -0.1700

Data Index: 2567748 (Open Web Text)

Max Activating Token Index: 84

Text #19

Max Range: 4.2542. Min Range: -4.2542

Max Act: 3.9559. Min Act: -0.1700

Data Index: 2506588 (Open Web Text)

Max Activating Token Index: 156