Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 789 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8748. Min Act: -0.1700

Data Index: 2129929 (Open Web Text)

Max Activating Token Index: 776

Text #1

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8781. Min Act: -0.1700

Data Index: 3897629 (Open Web Text)

Max Activating Token Index: 400

Text #2

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8306. Min Act: -0.1700

Data Index: 6788026 (Open Web Text)

Max Activating Token Index: 95

Text #3

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8573. Min Act: -0.1700

Data Index: 6358052 (Open Web Text)

Max Activating Token Index: 439

Text #4

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8332. Min Act: -0.1700

Data Index: 3262754 (Open Web Text)

Max Activating Token Index: 481

Text #5

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8068. Min Act: -0.1700

Data Index: 7082908 (Open Web Text)

Max Activating Token Index: 169

Text #6

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8008. Min Act: -0.1700

Data Index: 7492253 (Open Web Text)

Max Activating Token Index: 118

Text #7

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7865. Min Act: -0.1700

Data Index: 3383824 (Open Web Text)

Max Activating Token Index: 909

Text #8

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7797. Min Act: -0.1700

Data Index: 2417847 (Open Web Text)

Max Activating Token Index: 457

Text #9

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7778. Min Act: -0.1700

Data Index: 1325114 (Open Web Text)

Max Activating Token Index: 479

Text #10

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8114. Min Act: -0.1700

Data Index: 4415560 (Open Web Text)

Max Activating Token Index: 774

Text #11

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8194. Min Act: -0.1700

Data Index: 421801 (Open Web Text)

Max Activating Token Index: 397

Text #12

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7638. Min Act: -0.1700

Data Index: 5903584 (Open Web Text)

Max Activating Token Index: 417

Text #13

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7892. Min Act: -0.1700

Data Index: 6173841 (Open Web Text)

Max Activating Token Index: 425

Text #14

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8030. Min Act: -0.1700

Data Index: 6266384 (Open Web Text)

Max Activating Token Index: 598

Text #15

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7624. Min Act: -0.1700

Data Index: 2090224 (Open Web Text)

Max Activating Token Index: 309

Text #16

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7791. Min Act: -0.1700

Data Index: 3167395 (Open Web Text)

Max Activating Token Index: 968

Text #17

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.8066. Min Act: -0.1700

Data Index: 428078 (Open Web Text)

Max Activating Token Index: 813

Text #18

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7620. Min Act: -0.1700

Data Index: 6615848 (Open Web Text)

Max Activating Token Index: 649

Text #19

Max Range: 3.8781. Min Range: -3.8781

Max Act: 3.7971. Min Act: -0.1700

Data Index: 574404 (Open Web Text)

Max Activating Token Index: 365