Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2985 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.1595. Min Act: -0.1700

Data Index: 5689765 (Open Web Text)

Max Activating Token Index: 686

Text #1

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.1422. Min Act: -0.1700

Data Index: 4913251 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.1402. Min Act: -0.1700

Data Index: 5856552 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0902. Min Act: -0.1700

Data Index: 4968553 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0826. Min Act: -0.1700

Data Index: 4734332 (Open Web Text)

Max Activating Token Index: 10

Text #5

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0860. Min Act: -0.1700

Data Index: 8771212 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0644. Min Act: -0.1700

Data Index: 2415728 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0577. Min Act: -0.1700

Data Index: 3644097 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0562. Min Act: -0.1700

Data Index: 7983446 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0738. Min Act: -0.1700

Data Index: 7483193 (Open Web Text)

Max Activating Token Index: 975

Text #10

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0578. Min Act: -0.1700

Data Index: 5832960 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0591. Min Act: -0.1700

Data Index: 4057408 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0466. Min Act: -0.1700

Data Index: 7685559 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0417. Min Act: -0.1700

Data Index: 3186670 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0440. Min Act: -0.1700

Data Index: 3416384 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0204. Min Act: -0.1700

Data Index: 6402511 (Open Web Text)

Max Activating Token Index: 8

Text #16

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0005. Min Act: -0.1700

Data Index: 1646435 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0115. Min Act: -0.1700

Data Index: 5589833 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 4.1595. Min Range: -4.1595

Max Act: 3.9840. Min Act: -0.1700

Data Index: 3529064 (Open Web Text)

Max Activating Token Index: 8

Text #19

Max Range: 4.1595. Min Range: -4.1595

Max Act: 4.0134. Min Act: -0.1700

Data Index: 3985175 (Open Web Text)

Max Activating Token Index: 360