Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2073 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1856. Min Range: -4.1856

Max Act: 4.1856. Min Act: -0.1700

Data Index: 4944053 (Open Web Text)

Max Activating Token Index: 557

Text #1

Max Range: 4.1856. Min Range: -4.1856

Max Act: 4.0261. Min Act: -0.1700

Data Index: 4576543 (Open Web Text)

Max Activating Token Index: 242

Text #2

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.9718. Min Act: -0.1700

Data Index: 2090290 (Open Web Text)

Max Activating Token Index: 687

Text #3

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.9880. Min Act: -0.1700

Data Index: 6014398 (Open Web Text)

Max Activating Token Index: 490

Text #4

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.9129. Min Act: -0.1700

Data Index: 7728414 (Open Web Text)

Max Activating Token Index: 59

Text #5

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.9334. Min Act: -0.1700

Data Index: 517507 (Open Web Text)

Max Activating Token Index: 290

Text #6

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8726. Min Act: -0.1700

Data Index: 4585722 (Open Web Text)

Max Activating Token Index: 115

Text #7

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8989. Min Act: -0.1700

Data Index: 5197717 (Open Web Text)

Max Activating Token Index: 164

Text #8

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.9014. Min Act: -0.1700

Data Index: 3185496 (Open Web Text)

Max Activating Token Index: 515

Text #9

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8738. Min Act: -0.1700

Data Index: 5804193 (Open Web Text)

Max Activating Token Index: 932

Text #10

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8831. Min Act: -0.1700

Data Index: 7570868 (Open Web Text)

Max Activating Token Index: 123

Text #11

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8636. Min Act: -0.1700

Data Index: 4163379 (Open Web Text)

Max Activating Token Index: 589

Text #12

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8463. Min Act: -0.1700

Data Index: 5902286 (Open Web Text)

Max Activating Token Index: 413

Text #13

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8391. Min Act: -0.1700

Data Index: 2359502 (Open Web Text)

Max Activating Token Index: 787

Text #14

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8488. Min Act: -0.1700

Data Index: 1072695 (Open Web Text)

Max Activating Token Index: 546

Text #15

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8557. Min Act: -0.1700

Data Index: 6712421 (Open Web Text)

Max Activating Token Index: 574

Text #16

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8379. Min Act: -0.1700

Data Index: 6251002 (Open Web Text)

Max Activating Token Index: 45

Text #17

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8498. Min Act: -0.1700

Data Index: 4520879 (Open Web Text)

Max Activating Token Index: 907

Text #18

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8327. Min Act: -0.1700

Data Index: 7333222 (Open Web Text)

Max Activating Token Index: 271

Text #19

Max Range: 4.1856. Min Range: -4.1856

Max Act: 3.8077. Min Act: -0.1700

Data Index: 1924749 (Open Web Text)

Max Activating Token Index: 850