Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 902 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.8556. Min Act: -0.1700

Data Index: 2709776 (Open Web Text)

Max Activating Token Index: 200

Text #1

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.8452. Min Act: -0.1700

Data Index: 8501168 (Open Web Text)

Max Activating Token Index: 667

Text #2

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.8285. Min Act: -0.1700

Data Index: 1121460 (Open Web Text)

Max Activating Token Index: 92

Text #3

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.8049. Min Act: -0.1700

Data Index: 5811640 (Open Web Text)

Max Activating Token Index: 271

Text #4

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.8064. Min Act: -0.1700

Data Index: 3620039 (Open Web Text)

Max Activating Token Index: 550

Text #5

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7701. Min Act: -0.1700

Data Index: 5927569 (Open Web Text)

Max Activating Token Index: 333

Text #6

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7700. Min Act: -0.1700

Data Index: 5642698 (Open Web Text)

Max Activating Token Index: 414

Text #7

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7980. Min Act: -0.1700

Data Index: 3394571 (Open Web Text)

Max Activating Token Index: 561

Text #8

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7465. Min Act: -0.1700

Data Index: 435783 (Open Web Text)

Max Activating Token Index: 115

Text #9

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7246. Min Act: -0.1700

Data Index: 6802583 (Open Web Text)

Max Activating Token Index: 540

Text #10

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7570. Min Act: -0.1700

Data Index: 3710276 (Open Web Text)

Max Activating Token Index: 833

Text #11

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7495. Min Act: -0.1700

Data Index: 4955888 (Open Web Text)

Max Activating Token Index: 243

Text #12

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7544. Min Act: -0.1700

Data Index: 2428658 (Open Web Text)

Max Activating Token Index: 550

Text #13

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7608. Min Act: -0.1700

Data Index: 200151 (Open Web Text)

Max Activating Token Index: 468

Text #14

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7503. Min Act: -0.1700

Data Index: 5540205 (Open Web Text)

Max Activating Token Index: 286

Text #15

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.6947. Min Act: -0.1700

Data Index: 5190560 (Open Web Text)

Max Activating Token Index: 510

Text #16

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7149. Min Act: -0.1700

Data Index: 1642400 (Open Web Text)

Max Activating Token Index: 111

Text #17

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7046. Min Act: -0.1700

Data Index: 5076647 (Open Web Text)

Max Activating Token Index: 93

Text #18

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7420. Min Act: -0.1700

Data Index: 1812665 (Open Web Text)

Max Activating Token Index: 161

Text #19

Max Range: 3.8556. Min Range: -3.8556

Max Act: 3.7379. Min Act: -0.1700

Data Index: 2733358 (Open Web Text)

Max Activating Token Index: 557