Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1802 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.1188. Min Act: -0.1700

Data Index: 889098 (Open Web Text)

Max Activating Token Index: 11

Text #1

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.1089. Min Act: -0.1700

Data Index: 1781977 (Open Web Text)

Max Activating Token Index: 38

Text #2

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0452. Min Act: -0.1700

Data Index: 1918654 (Open Web Text)

Max Activating Token Index: 794

Text #3

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0462. Min Act: -0.1700

Data Index: 6243713 (Open Web Text)

Max Activating Token Index: 22

Text #4

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0447. Min Act: -0.1700

Data Index: 1199114 (Open Web Text)

Max Activating Token Index: 23

Text #5

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0076. Min Act: -0.1700

Data Index: 5718640 (Open Web Text)

Max Activating Token Index: 921

Text #6

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0115. Min Act: -0.1700

Data Index: 6618286 (Open Web Text)

Max Activating Token Index: 19

Text #7

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0024. Min Act: -0.1700

Data Index: 6166062 (Open Web Text)

Max Activating Token Index: 37

Text #8

Max Range: 6.1188. Min Range: -6.1188

Max Act: 6.0263. Min Act: -0.1700

Data Index: 4511197 (Open Web Text)

Max Activating Token Index: 15

Text #9

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9772. Min Act: -0.1700

Data Index: 3680753 (Open Web Text)

Max Activating Token Index: 27

Text #10

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9778. Min Act: -0.1700

Data Index: 4845284 (Open Web Text)

Max Activating Token Index: 34

Text #11

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9754. Min Act: -0.1700

Data Index: 591054 (Open Web Text)

Max Activating Token Index: 21

Text #12

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9975. Min Act: -0.1700

Data Index: 4971273 (Open Web Text)

Max Activating Token Index: 30

Text #13

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9864. Min Act: -0.1700

Data Index: 2232500 (Open Web Text)

Max Activating Token Index: 21

Text #14

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9626. Min Act: -0.1700

Data Index: 82823 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9715. Min Act: -0.1700

Data Index: 5611474 (Open Web Text)

Max Activating Token Index: 17

Text #16

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9990. Min Act: -0.1700

Data Index: 1891486 (Open Web Text)

Max Activating Token Index: 10

Text #17

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9713. Min Act: -0.1700

Data Index: 30254 (Open Web Text)

Max Activating Token Index: 25

Text #18

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9395. Min Act: -0.1700

Data Index: 5197152 (Open Web Text)

Max Activating Token Index: 38

Text #19

Max Range: 6.1188. Min Range: -6.1188

Max Act: 5.9599. Min Act: -0.1700

Data Index: 4199826 (Open Web Text)

Max Activating Token Index: 24