Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1504 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1166. Min Range: -4.1166

Max Act: 4.1166. Min Act: -0.1700

Data Index: 4187348 (Open Web Text)

Max Activating Token Index: 551

Text #1

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.9906. Min Act: -0.1700

Data Index: 515669 (Open Web Text)

Max Activating Token Index: 186

Text #2

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.9376. Min Act: -0.1700

Data Index: 4263645 (Open Web Text)

Max Activating Token Index: 718

Text #3

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.9210. Min Act: -0.1700

Data Index: 587048 (Open Web Text)

Max Activating Token Index: 876

Text #4

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8504. Min Act: -0.1700

Data Index: 447979 (Open Web Text)

Max Activating Token Index: 894

Text #5

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8848. Min Act: -0.1700

Data Index: 3478898 (Open Web Text)

Max Activating Token Index: 984

Text #6

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8655. Min Act: -0.1700

Data Index: 150926 (Open Web Text)

Max Activating Token Index: 460

Text #7

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8570. Min Act: -0.1700

Data Index: 4022570 (Open Web Text)

Max Activating Token Index: 760

Text #8

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8121. Min Act: -0.1700

Data Index: 53744 (Open Web Text)

Max Activating Token Index: 923

Text #9

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8124. Min Act: -0.1700

Data Index: 3165056 (Open Web Text)

Max Activating Token Index: 636

Text #10

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8187. Min Act: -0.1700

Data Index: 1352253 (Open Web Text)

Max Activating Token Index: 37

Text #11

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7992. Min Act: -0.1700

Data Index: 1979371 (Open Web Text)

Max Activating Token Index: 546

Text #12

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8147. Min Act: -0.1700

Data Index: 414371 (Open Web Text)

Max Activating Token Index: 215

Text #13

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8209. Min Act: -0.1700

Data Index: 8214268 (Open Web Text)

Max Activating Token Index: 647

Text #14

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.8045. Min Act: -0.1700

Data Index: 6800958 (Open Web Text)

Max Activating Token Index: 378

Text #15

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7911. Min Act: -0.1700

Data Index: 1159028 (Open Web Text)

Max Activating Token Index: 792

Text #16

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7757. Min Act: -0.1700

Data Index: 3691294 (Open Web Text)

Max Activating Token Index: 489

Text #17

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7857. Min Act: -0.1700

Data Index: 3994995 (Open Web Text)

Max Activating Token Index: 144

Text #18

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7572. Min Act: -0.1700

Data Index: 3511783 (Open Web Text)

Max Activating Token Index: 909

Text #19

Max Range: 4.1166. Min Range: -4.1166

Max Act: 3.7684. Min Act: -0.1700

Data Index: 3852546 (Open Web Text)

Max Activating Token Index: 182