Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2496 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0280. Min Range: -4.0280

Max Act: 4.0280. Min Act: -0.1700

Data Index: 6020446 (Open Web Text)

Max Activating Token Index: 425

Text #1

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.9708. Min Act: -0.1700

Data Index: 6020447 (Open Web Text)

Max Activating Token Index: 946

Text #2

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8784. Min Act: -0.1700

Data Index: 7319304 (Open Web Text)

Max Activating Token Index: 526

Text #3

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.9277. Min Act: -0.1700

Data Index: 8252688 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.9163. Min Act: -0.1700

Data Index: 5931749 (Open Web Text)

Max Activating Token Index: 272

Text #5

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.9052. Min Act: -0.1700

Data Index: 6200034 (Open Web Text)

Max Activating Token Index: 641

Text #6

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.9133. Min Act: -0.1700

Data Index: 6297495 (Open Web Text)

Max Activating Token Index: 670

Text #7

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8837. Min Act: -0.1700

Data Index: 7918443 (Open Web Text)

Max Activating Token Index: 546

Text #8

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8687. Min Act: -0.1700

Data Index: 2577697 (Open Web Text)

Max Activating Token Index: 835

Text #9

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8542. Min Act: -0.1700

Data Index: 1565449 (Open Web Text)

Max Activating Token Index: 607

Text #10

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8515. Min Act: -0.1700

Data Index: 7298603 (Open Web Text)

Max Activating Token Index: 1002

Text #11

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8488. Min Act: -0.1700

Data Index: 5416225 (Open Web Text)

Max Activating Token Index: 400

Text #12

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8545. Min Act: -0.1700

Data Index: 544933 (Open Web Text)

Max Activating Token Index: 467

Text #13

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8555. Min Act: -0.1700

Data Index: 5376944 (Open Web Text)

Max Activating Token Index: 366

Text #14

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8446. Min Act: -0.1700

Data Index: 963802 (Open Web Text)

Max Activating Token Index: 315

Text #15

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8613. Min Act: -0.1700

Data Index: 2397182 (Open Web Text)

Max Activating Token Index: 760

Text #16

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8515. Min Act: -0.1700

Data Index: 6856204 (Open Web Text)

Max Activating Token Index: 640

Text #17

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8551. Min Act: -0.1700

Data Index: 1344966 (Open Web Text)

Max Activating Token Index: 355

Text #18

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8594. Min Act: -0.1700

Data Index: 6076719 (Open Web Text)

Max Activating Token Index: 913

Text #19

Max Range: 4.0280. Min Range: -4.0280

Max Act: 3.8631. Min Act: -0.1700

Data Index: 1161729 (Open Web Text)

Max Activating Token Index: 17