Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1807 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.3221. Min Act: -0.1700

Data Index: 1697839 (Open Web Text)

Max Activating Token Index: 335

Text #1

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.3130. Min Act: -0.1700

Data Index: 665934 (Open Web Text)

Max Activating Token Index: 804

Text #2

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.3252. Min Act: -0.1700

Data Index: 8332908 (Open Web Text)

Max Activating Token Index: 792

Text #3

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.1505. Min Act: -0.1700

Data Index: 7340670 (Open Web Text)

Max Activating Token Index: 154

Text #4

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.1317. Min Act: -0.1700

Data Index: 1718301 (Open Web Text)

Max Activating Token Index: 120

Text #5

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.1174. Min Act: -0.1700

Data Index: 3470722 (Open Web Text)

Max Activating Token Index: 1009

Text #6

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.1011. Min Act: -0.1700

Data Index: 4203470 (Open Web Text)

Max Activating Token Index: 115

Text #7

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0640. Min Act: -0.1700

Data Index: 1880935 (Open Web Text)

Max Activating Token Index: 827

Text #8

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0309. Min Act: -0.1700

Data Index: 4792877 (Open Web Text)

Max Activating Token Index: 303

Text #9

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.1007. Min Act: -0.1700

Data Index: 1120430 (Open Web Text)

Max Activating Token Index: 619

Text #10

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0901. Min Act: -0.1700

Data Index: 3529410 (Open Web Text)

Max Activating Token Index: 644

Text #11

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0267. Min Act: -0.1700

Data Index: 5561407 (Open Web Text)

Max Activating Token Index: 216

Text #12

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0654. Min Act: -0.1700

Data Index: 6918134 (Open Web Text)

Max Activating Token Index: 394

Text #13

Max Range: 5.3252. Min Range: -5.3252

Max Act: 4.9815. Min Act: -0.1700

Data Index: 7279409 (Open Web Text)

Max Activating Token Index: 553

Text #14

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0060. Min Act: -0.1700

Data Index: 6918275 (Open Web Text)

Max Activating Token Index: 378

Text #15

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0023. Min Act: -0.1700

Data Index: 2034091 (Open Web Text)

Max Activating Token Index: 768

Text #16

Max Range: 5.3252. Min Range: -5.3252

Max Act: 4.9786. Min Act: -0.1700

Data Index: 276302 (Open Web Text)

Max Activating Token Index: 917

Text #17

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0270. Min Act: -0.1700

Data Index: 1572046 (Open Web Text)

Max Activating Token Index: 793

Text #18

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0058. Min Act: -0.1700

Data Index: 7439866 (Open Web Text)

Max Activating Token Index: 396

Text #19

Max Range: 5.3252. Min Range: -5.3252

Max Act: 5.0159. Min Act: -0.1700

Data Index: 4746784 (Open Web Text)

Max Activating Token Index: 917