Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2522 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.5820. Min Act: -0.1700

Data Index: 8458416 (Open Web Text)

Max Activating Token Index: 422

Text #1

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.6183. Min Act: -0.1700

Data Index: 8171352 (Open Web Text)

Max Activating Token Index: 650

Text #2

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.5082. Min Act: -0.1700

Data Index: 5503619 (Open Web Text)

Max Activating Token Index: 947

Text #3

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.4730. Min Act: -0.1700

Data Index: 986172 (Open Web Text)

Max Activating Token Index: 439

Text #4

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.4776. Min Act: -0.1700

Data Index: 3850750 (Open Web Text)

Max Activating Token Index: 791

Text #5

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.4088. Min Act: -0.1700

Data Index: 7417488 (Open Web Text)

Max Activating Token Index: 940

Text #6

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3913. Min Act: -0.1700

Data Index: 1984058 (Open Web Text)

Max Activating Token Index: 690

Text #7

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3581. Min Act: -0.1700

Data Index: 774393 (Open Web Text)

Max Activating Token Index: 400

Text #8

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3856. Min Act: -0.1700

Data Index: 6732293 (Open Web Text)

Max Activating Token Index: 474

Text #9

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3591. Min Act: -0.1700

Data Index: 6114348 (Open Web Text)

Max Activating Token Index: 253

Text #10

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3827. Min Act: -0.1700

Data Index: 100976 (Open Web Text)

Max Activating Token Index: 892

Text #11

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3679. Min Act: -0.1700

Data Index: 5637952 (Open Web Text)

Max Activating Token Index: 111

Text #12

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3290. Min Act: -0.1700

Data Index: 4529569 (Open Web Text)

Max Activating Token Index: 557

Text #13

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3579. Min Act: -0.1700

Data Index: 1114334 (Open Web Text)

Max Activating Token Index: 388

Text #14

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3535. Min Act: -0.1700

Data Index: 7405561 (Open Web Text)

Max Activating Token Index: 557

Text #15

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3819. Min Act: -0.1700

Data Index: 7213321 (Open Web Text)

Max Activating Token Index: 442

Text #16

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.2925. Min Act: -0.1700

Data Index: 3647040 (Open Web Text)

Max Activating Token Index: 349

Text #17

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.2870. Min Act: -0.1700

Data Index: 903715 (Open Web Text)

Max Activating Token Index: 434

Text #18

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3556. Min Act: -0.1700

Data Index: 5877206 (Open Web Text)

Max Activating Token Index: 317

Text #19

Max Range: 4.6183. Min Range: -4.6183

Max Act: 4.3272. Min Act: -0.1700

Data Index: 5982614 (Open Web Text)

Max Activating Token Index: 919