Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2553 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.5687. Min Act: -0.1700

Data Index: 3087499 (Open Web Text)

Max Activating Token Index: 76

Text #1

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.5338. Min Act: -0.1700

Data Index: 711184 (Open Web Text)

Max Activating Token Index: 354

Text #2

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4612. Min Act: -0.1700

Data Index: 7362267 (Open Web Text)

Max Activating Token Index: 98

Text #3

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4366. Min Act: -0.1700

Data Index: 7643141 (Open Web Text)

Max Activating Token Index: 495

Text #4

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4488. Min Act: -0.1700

Data Index: 2268294 (Open Web Text)

Max Activating Token Index: 872

Text #5

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4420. Min Act: -0.1700

Data Index: 2177218 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4220. Min Act: -0.1700

Data Index: 1095405 (Open Web Text)

Max Activating Token Index: 1001

Text #7

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3912. Min Act: -0.1700

Data Index: 678161 (Open Web Text)

Max Activating Token Index: 996

Text #8

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.4198. Min Act: -0.1700

Data Index: 2642593 (Open Web Text)

Max Activating Token Index: 961

Text #9

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3949. Min Act: -0.1700

Data Index: 5729815 (Open Web Text)

Max Activating Token Index: 665

Text #10

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3873. Min Act: -0.1700

Data Index: 1387486 (Open Web Text)

Max Activating Token Index: 688

Text #11

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3874. Min Act: -0.1700

Data Index: 4006200 (Open Web Text)

Max Activating Token Index: 357

Text #12

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3746. Min Act: -0.1700

Data Index: 3960852 (Open Web Text)

Max Activating Token Index: 743

Text #13

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3883. Min Act: -0.1700

Data Index: 6533927 (Open Web Text)

Max Activating Token Index: 928

Text #14

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3959. Min Act: -0.1700

Data Index: 3145494 (Open Web Text)

Max Activating Token Index: 945

Text #15

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3902. Min Act: -0.1700

Data Index: 264506 (Open Web Text)

Max Activating Token Index: 307

Text #16

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3743. Min Act: -0.1700

Data Index: 1355381 (Open Web Text)

Max Activating Token Index: 1022

Text #17

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3742. Min Act: -0.1700

Data Index: 5119326 (Open Web Text)

Max Activating Token Index: 326

Text #18

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3724. Min Act: -0.1700

Data Index: 4911438 (Open Web Text)

Max Activating Token Index: 429

Text #19

Max Range: 4.5687. Min Range: -4.5687

Max Act: 4.3920. Min Act: -0.1700

Data Index: 4677297 (Open Web Text)

Max Activating Token Index: 656