Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 27 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8959. Min Act: -0.1700

Data Index: 6887402 (Open Web Text)

Max Activating Token Index: 956

Text #1

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8752. Min Act: -0.1700

Data Index: 5894188 (Open Web Text)

Max Activating Token Index: 296

Text #2

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8545. Min Act: -0.1700

Data Index: 273271 (Open Web Text)

Max Activating Token Index: 934

Text #3

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8756. Min Act: -0.1700

Data Index: 5276718 (Open Web Text)

Max Activating Token Index: 715

Text #4

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8753. Min Act: -0.1700

Data Index: 8464373 (Open Web Text)

Max Activating Token Index: 967

Text #5

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8669. Min Act: -0.1700

Data Index: 1105225 (Open Web Text)

Max Activating Token Index: 607

Text #6

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8588. Min Act: -0.1700

Data Index: 5527377 (Open Web Text)

Max Activating Token Index: 782

Text #7

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8709. Min Act: -0.1700

Data Index: 1913418 (Open Web Text)

Max Activating Token Index: 461

Text #8

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8498. Min Act: -0.1700

Data Index: 7893308 (Open Web Text)

Max Activating Token Index: 734

Text #9

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8615. Min Act: -0.1700

Data Index: 2582054 (Open Web Text)

Max Activating Token Index: 401

Text #10

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8585. Min Act: -0.1700

Data Index: 3965025 (Open Web Text)

Max Activating Token Index: 438

Text #11

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8382. Min Act: -0.1700

Data Index: 3594310 (Open Web Text)

Max Activating Token Index: 605

Text #12

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8452. Min Act: -0.1700

Data Index: 6672696 (Open Web Text)

Max Activating Token Index: 189

Text #13

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8319. Min Act: -0.1700

Data Index: 7989096 (Open Web Text)

Max Activating Token Index: 575

Text #14

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8318. Min Act: -0.1700

Data Index: 4419129 (Open Web Text)

Max Activating Token Index: 437

Text #15

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8228. Min Act: -0.1700

Data Index: 3413017 (Open Web Text)

Max Activating Token Index: 950

Text #16

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.8189. Min Act: -0.1700

Data Index: 3233487 (Open Web Text)

Max Activating Token Index: 845

Text #17

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.7802. Min Act: -0.1700

Data Index: 1906058 (Open Web Text)

Max Activating Token Index: 768

Text #18

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.7816. Min Act: -0.1700

Data Index: 4807661 (Open Web Text)

Max Activating Token Index: 484

Text #19

Max Range: 3.8959. Min Range: -3.8959

Max Act: 3.7571. Min Act: -0.1700

Data Index: 5096369 (Open Web Text)

Max Activating Token Index: 669