Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 622 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.2440. Min Act: -0.1700

Data Index: 3492281 (Open Web Text)

Max Activating Token Index: 114

Text #1

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.2009. Min Act: -0.1700

Data Index: 8530438 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.2276. Min Act: -0.1700

Data Index: 6570216 (Open Web Text)

Max Activating Token Index: 437

Text #3

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.2016. Min Act: -0.1700

Data Index: 7848266 (Open Web Text)

Max Activating Token Index: 583

Text #4

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1648. Min Act: -0.1700

Data Index: 7626124 (Open Web Text)

Max Activating Token Index: 236

Text #5

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1522. Min Act: -0.1700

Data Index: 8333185 (Open Web Text)

Max Activating Token Index: 664

Text #6

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1782. Min Act: -0.1700

Data Index: 5018537 (Open Web Text)

Max Activating Token Index: 997

Text #7

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1585. Min Act: -0.1700

Data Index: 7196662 (Open Web Text)

Max Activating Token Index: 10

Text #8

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1367. Min Act: -0.1700

Data Index: 623344 (Open Web Text)

Max Activating Token Index: 22

Text #9

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1127. Min Act: -0.1700

Data Index: 2855649 (Open Web Text)

Max Activating Token Index: 639

Text #10

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1014. Min Act: -0.1700

Data Index: 4528177 (Open Web Text)

Max Activating Token Index: 227

Text #11

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1293. Min Act: -0.1700

Data Index: 3671722 (Open Web Text)

Max Activating Token Index: 41

Text #12

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1349. Min Act: -0.1700

Data Index: 1136481 (Open Web Text)

Max Activating Token Index: 77

Text #13

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1316. Min Act: -0.1700

Data Index: 8632948 (Open Web Text)

Max Activating Token Index: 168

Text #14

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1071. Min Act: -0.1700

Data Index: 7624772 (Open Web Text)

Max Activating Token Index: 137

Text #15

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1057. Min Act: -0.1700

Data Index: 1042032 (Open Web Text)

Max Activating Token Index: 774

Text #16

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1015. Min Act: -0.1700

Data Index: 7782869 (Open Web Text)

Max Activating Token Index: 556

Text #17

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.0999. Min Act: -0.1700

Data Index: 6427787 (Open Web Text)

Max Activating Token Index: 444

Text #18

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.1125. Min Act: -0.1700

Data Index: 5040574 (Open Web Text)

Max Activating Token Index: 128

Text #19

Max Range: 4.2440. Min Range: -4.2440

Max Act: 4.0926. Min Act: -0.1700

Data Index: 3552211 (Open Web Text)

Max Activating Token Index: 98