Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 579 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0201. Min Range: -4.0201

Max Act: 4.0201. Min Act: -0.1700

Data Index: 2483928 (Open Web Text)

Max Activating Token Index: 917

Text #1

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9950. Min Act: -0.1700

Data Index: 7125310 (Open Web Text)

Max Activating Token Index: 946

Text #2

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9904. Min Act: -0.1700

Data Index: 7253784 (Open Web Text)

Max Activating Token Index: 357

Text #3

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9757. Min Act: -0.1700

Data Index: 1282175 (Open Web Text)

Max Activating Token Index: 752

Text #4

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9430. Min Act: -0.1700

Data Index: 4263901 (Open Web Text)

Max Activating Token Index: 828

Text #5

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9319. Min Act: -0.1700

Data Index: 7355814 (Open Web Text)

Max Activating Token Index: 569

Text #6

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9306. Min Act: -0.1700

Data Index: 750632 (Open Web Text)

Max Activating Token Index: 568

Text #7

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8914. Min Act: -0.1700

Data Index: 3315381 (Open Web Text)

Max Activating Token Index: 705

Text #8

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.9257. Min Act: -0.1700

Data Index: 3990592 (Open Web Text)

Max Activating Token Index: 468

Text #9

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8967. Min Act: -0.1700

Data Index: 4153554 (Open Web Text)

Max Activating Token Index: 825

Text #10

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8998. Min Act: -0.1700

Data Index: 4386 (Open Web Text)

Max Activating Token Index: 824

Text #11

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8927. Min Act: -0.1700

Data Index: 3450555 (Open Web Text)

Max Activating Token Index: 852

Text #12

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8891. Min Act: -0.1700

Data Index: 2928738 (Open Web Text)

Max Activating Token Index: 361

Text #13

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8604. Min Act: -0.1700

Data Index: 5170449 (Open Web Text)

Max Activating Token Index: 621

Text #14

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8444. Min Act: -0.1700

Data Index: 6608282 (Open Web Text)

Max Activating Token Index: 295

Text #15

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8568. Min Act: -0.1700

Data Index: 533980 (Open Web Text)

Max Activating Token Index: 273

Text #16

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8493. Min Act: -0.1700

Data Index: 4984861 (Open Web Text)

Max Activating Token Index: 125

Text #17

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8451. Min Act: -0.1700

Data Index: 1132403 (Open Web Text)

Max Activating Token Index: 227

Text #18

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8671. Min Act: -0.1700

Data Index: 718085 (Open Web Text)

Max Activating Token Index: 657

Text #19

Max Range: 4.0201. Min Range: -4.0201

Max Act: 3.8593. Min Act: -0.1700

Data Index: 2397489 (Open Web Text)

Max Activating Token Index: 816