Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 316 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.2276. Min Act: -0.1700

Data Index: 7472083 (Open Web Text)

Max Activating Token Index: 23

Text #1

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.2312. Min Act: -0.1700

Data Index: 6606128 (Open Web Text)

Max Activating Token Index: 52

Text #2

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.2080. Min Act: -0.1700

Data Index: 1590903 (Open Web Text)

Max Activating Token Index: 1013

Text #3

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.2002. Min Act: -0.1700

Data Index: 8272649 (Open Web Text)

Max Activating Token Index: 99

Text #4

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1366. Min Act: -0.1700

Data Index: 5362744 (Open Web Text)

Max Activating Token Index: 109

Text #5

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1498. Min Act: -0.1700

Data Index: 1718293 (Open Web Text)

Max Activating Token Index: 45

Text #6

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.0920. Min Act: -0.1700

Data Index: 7037473 (Open Web Text)

Max Activating Token Index: 203

Text #7

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1464. Min Act: -0.1700

Data Index: 4243995 (Open Web Text)

Max Activating Token Index: 999

Text #8

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1333. Min Act: -0.1700

Data Index: 6725511 (Open Web Text)

Max Activating Token Index: 772

Text #9

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1295. Min Act: -0.1700

Data Index: 7566959 (Open Web Text)

Max Activating Token Index: 31

Text #10

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1103. Min Act: -0.1700

Data Index: 6570132 (Open Web Text)

Max Activating Token Index: 247

Text #11

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1182. Min Act: -0.1700

Data Index: 176546 (Open Web Text)

Max Activating Token Index: 1001

Text #12

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1078. Min Act: -0.1700

Data Index: 8417711 (Open Web Text)

Max Activating Token Index: 544

Text #13

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1271. Min Act: -0.1700

Data Index: 7454450 (Open Web Text)

Max Activating Token Index: 145

Text #14

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1435. Min Act: -0.1700

Data Index: 3860586 (Open Web Text)

Max Activating Token Index: 237

Text #15

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1235. Min Act: -0.1700

Data Index: 3106014 (Open Web Text)

Max Activating Token Index: 23

Text #16

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1221. Min Act: -0.1700

Data Index: 5362739 (Open Web Text)

Max Activating Token Index: 124

Text #17

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.1325. Min Act: -0.1700

Data Index: 5052239 (Open Web Text)

Max Activating Token Index: 520

Text #18

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.0908. Min Act: -0.1700

Data Index: 916980 (Open Web Text)

Max Activating Token Index: 23

Text #19

Max Range: 4.2312. Min Range: -4.2312

Max Act: 4.0546. Min Act: -0.1700

Data Index: 2756147 (Open Web Text)

Max Activating Token Index: 137