Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2208 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8927. Min Act: -0.1700

Data Index: 2928753 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8463. Min Act: -0.1700

Data Index: 5227267 (Open Web Text)

Max Activating Token Index: 451

Text #2

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8536. Min Act: -0.1700

Data Index: 3137945 (Open Web Text)

Max Activating Token Index: 126

Text #3

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8583. Min Act: -0.1700

Data Index: 728704 (Open Web Text)

Max Activating Token Index: 252

Text #4

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8659. Min Act: -0.1700

Data Index: 2452336 (Open Web Text)

Max Activating Token Index: 100

Text #5

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8702. Min Act: -0.1700

Data Index: 215605 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8204. Min Act: -0.1700

Data Index: 7021441 (Open Web Text)

Max Activating Token Index: 392

Text #7

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8230. Min Act: -0.1700

Data Index: 4313938 (Open Web Text)

Max Activating Token Index: 536

Text #8

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8249. Min Act: -0.1700

Data Index: 3626411 (Open Web Text)

Max Activating Token Index: 140

Text #9

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7980. Min Act: -0.1700

Data Index: 4310895 (Open Web Text)

Max Activating Token Index: 531

Text #10

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8100. Min Act: -0.1700

Data Index: 3856848 (Open Web Text)

Max Activating Token Index: 445

Text #11

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7818. Min Act: -0.1700

Data Index: 5400394 (Open Web Text)

Max Activating Token Index: 481

Text #12

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7876. Min Act: -0.1700

Data Index: 562020 (Open Web Text)

Max Activating Token Index: 157

Text #13

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7860. Min Act: -0.1700

Data Index: 6935009 (Open Web Text)

Max Activating Token Index: 958

Text #14

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8209. Min Act: -0.1700

Data Index: 6274736 (Open Web Text)

Max Activating Token Index: 208

Text #15

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8024. Min Act: -0.1700

Data Index: 4889154 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7817. Min Act: -0.1700

Data Index: 1115516 (Open Web Text)

Max Activating Token Index: 219

Text #17

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.7864. Min Act: -0.1700

Data Index: 3178309 (Open Web Text)

Max Activating Token Index: 155

Text #18

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8006. Min Act: -0.1700

Data Index: 5361813 (Open Web Text)

Max Activating Token Index: 46

Text #19

Max Range: 4.8927. Min Range: -4.8927

Max Act: 4.8058. Min Act: -0.1700

Data Index: 4988032 (Open Web Text)

Max Activating Token Index: 122