Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2717 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8709. Min Act: -0.1700

Data Index: 5847893 (Open Web Text)

Max Activating Token Index: 655

Text #1

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8805. Min Act: -0.1700

Data Index: 2480522 (Open Web Text)

Max Activating Token Index: 486

Text #2

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.9034. Min Act: -0.1700

Data Index: 1864583 (Open Web Text)

Max Activating Token Index: 401

Text #3

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8577. Min Act: -0.1700

Data Index: 6259647 (Open Web Text)

Max Activating Token Index: 980

Text #4

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8349. Min Act: -0.1700

Data Index: 5381906 (Open Web Text)

Max Activating Token Index: 617

Text #5

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8456. Min Act: -0.1700

Data Index: 5235143 (Open Web Text)

Max Activating Token Index: 983

Text #6

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8033. Min Act: -0.1700

Data Index: 7248566 (Open Web Text)

Max Activating Token Index: 450

Text #7

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7865. Min Act: -0.1700

Data Index: 8610607 (Open Web Text)

Max Activating Token Index: 220

Text #8

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7913. Min Act: -0.1700

Data Index: 1310555 (Open Web Text)

Max Activating Token Index: 268

Text #9

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.8044. Min Act: -0.1700

Data Index: 7310647 (Open Web Text)

Max Activating Token Index: 55

Text #10

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7858. Min Act: -0.1700

Data Index: 385470 (Open Web Text)

Max Activating Token Index: 927

Text #11

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7781. Min Act: -0.1700

Data Index: 4917622 (Open Web Text)

Max Activating Token Index: 729

Text #12

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7836. Min Act: -0.1700

Data Index: 1650418 (Open Web Text)

Max Activating Token Index: 651

Text #13

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7815. Min Act: -0.1700

Data Index: 797598 (Open Web Text)

Max Activating Token Index: 162

Text #14

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7815. Min Act: -0.1700

Data Index: 2995471 (Open Web Text)

Max Activating Token Index: 372

Text #15

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7365. Min Act: -0.1700

Data Index: 4655318 (Open Web Text)

Max Activating Token Index: 677

Text #16

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7569. Min Act: -0.1700

Data Index: 6691618 (Open Web Text)

Max Activating Token Index: 898

Text #17

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7396. Min Act: -0.1700

Data Index: 8151588 (Open Web Text)

Max Activating Token Index: 295

Text #18

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7497. Min Act: -0.1700

Data Index: 4540807 (Open Web Text)

Max Activating Token Index: 288

Text #19

Max Range: 3.9034. Min Range: -3.9034

Max Act: 3.7394. Min Act: -0.1700

Data Index: 1341015 (Open Web Text)

Max Activating Token Index: 591