Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1718 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.1283. Min Act: -0.1700

Data Index: 3975528 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.0570. Min Act: -0.1700

Data Index: 573089 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.0617. Min Act: -0.1700

Data Index: 8440808 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.0187. Min Act: -0.1700

Data Index: 8480147 (Open Web Text)

Max Activating Token Index: 660

Text #4

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.0020. Min Act: -0.1700

Data Index: 3158191 (Open Web Text)

Max Activating Token Index: 765

Text #5

Max Range: 5.1283. Min Range: -5.1283

Max Act: 5.0229. Min Act: -0.1700

Data Index: 2593270 (Open Web Text)

Max Activating Token Index: 653

Text #6

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9902. Min Act: -0.1700

Data Index: 3496313 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9809. Min Act: -0.1700

Data Index: 4744414 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9772. Min Act: -0.1700

Data Index: 4500373 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9651. Min Act: -0.1700

Data Index: 1464057 (Open Web Text)

Max Activating Token Index: 449

Text #10

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9276. Min Act: -0.1700

Data Index: 4179832 (Open Web Text)

Max Activating Token Index: 669

Text #11

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9185. Min Act: -0.1700

Data Index: 2254222 (Open Web Text)

Max Activating Token Index: 654

Text #12

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9212. Min Act: -0.1700

Data Index: 3978729 (Open Web Text)

Max Activating Token Index: 631

Text #13

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9282. Min Act: -0.1700

Data Index: 2943697 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9407. Min Act: -0.1700

Data Index: 5024092 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9351. Min Act: -0.1700

Data Index: 5393505 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9152. Min Act: -0.1700

Data Index: 6137590 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9228. Min Act: -0.1700

Data Index: 7284596 (Open Web Text)

Max Activating Token Index: 828

Text #18

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9201. Min Act: -0.1700

Data Index: 3901405 (Open Web Text)

Max Activating Token Index: 23

Text #19

Max Range: 5.1283. Min Range: -5.1283

Max Act: 4.9420. Min Act: -0.1700

Data Index: 1518521 (Open Web Text)

Max Activating Token Index: 4