Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 644 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.9059. Min Act: -0.1700

Data Index: 5837085 (Open Web Text)

Max Activating Token Index: 741

Text #1

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6959. Min Act: -0.1700

Data Index: 2032064 (Open Web Text)

Max Activating Token Index: 629

Text #2

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.7118. Min Act: -0.1700

Data Index: 5409637 (Open Web Text)

Max Activating Token Index: 631

Text #3

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6918. Min Act: -0.1700

Data Index: 2032058 (Open Web Text)

Max Activating Token Index: 630

Text #4

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6035. Min Act: -0.1700

Data Index: 5837079 (Open Web Text)

Max Activating Token Index: 845

Text #5

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5930. Min Act: -0.1700

Data Index: 5837070 (Open Web Text)

Max Activating Token Index: 362

Text #6

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6272. Min Act: -0.1700

Data Index: 232271 (Open Web Text)

Max Activating Token Index: 661

Text #7

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6064. Min Act: -0.1700

Data Index: 7376019 (Open Web Text)

Max Activating Token Index: 952

Text #8

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6058. Min Act: -0.1700

Data Index: 2576936 (Open Web Text)

Max Activating Token Index: 172

Text #9

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6060. Min Act: -0.1700

Data Index: 1553048 (Open Web Text)

Max Activating Token Index: 422

Text #10

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5989. Min Act: -0.1700

Data Index: 5837092 (Open Web Text)

Max Activating Token Index: 259

Text #11

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6453. Min Act: -0.1700

Data Index: 7146081 (Open Web Text)

Max Activating Token Index: 119

Text #12

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5675. Min Act: -0.1700

Data Index: 5837068 (Open Web Text)

Max Activating Token Index: 144

Text #13

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5525. Min Act: -0.1700

Data Index: 3023979 (Open Web Text)

Max Activating Token Index: 516

Text #14

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.6095. Min Act: -0.1700

Data Index: 5837077 (Open Web Text)

Max Activating Token Index: 565

Text #15

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5620. Min Act: -0.1700

Data Index: 5837080 (Open Web Text)

Max Activating Token Index: 607

Text #16

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5614. Min Act: -0.1700

Data Index: 2032066 (Open Web Text)

Max Activating Token Index: 947

Text #17

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5599. Min Act: -0.1700

Data Index: 232274 (Open Web Text)

Max Activating Token Index: 531

Text #18

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.5634. Min Act: -0.1700

Data Index: 527827 (Open Web Text)

Max Activating Token Index: 600

Text #19

Max Range: 4.9059. Min Range: -4.9059

Max Act: 4.4747. Min Act: -0.1700

Data Index: 1791140 (Open Web Text)

Max Activating Token Index: 542