Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2940 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.5517. Min Act: -0.1700

Data Index: 2105955 (Open Web Text)

Max Activating Token Index: 119

Text #1

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.5028. Min Act: -0.1700

Data Index: 5324771 (Open Web Text)

Max Activating Token Index: 456

Text #2

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.4117. Min Act: -0.1700

Data Index: 6574352 (Open Web Text)

Max Activating Token Index: 613

Text #3

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3674. Min Act: -0.1700

Data Index: 5474275 (Open Web Text)

Max Activating Token Index: 866

Text #4

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3851. Min Act: -0.1700

Data Index: 7716891 (Open Web Text)

Max Activating Token Index: 933

Text #5

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3626. Min Act: -0.1700

Data Index: 4402967 (Open Web Text)

Max Activating Token Index: 879

Text #6

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3614. Min Act: -0.1700

Data Index: 4315398 (Open Web Text)

Max Activating Token Index: 748

Text #7

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3619. Min Act: -0.1700

Data Index: 621746 (Open Web Text)

Max Activating Token Index: 790

Text #8

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3344. Min Act: -0.1700

Data Index: 3565645 (Open Web Text)

Max Activating Token Index: 998

Text #9

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3572. Min Act: -0.1700

Data Index: 8084527 (Open Web Text)

Max Activating Token Index: 917

Text #10

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3668. Min Act: -0.1700

Data Index: 8036326 (Open Web Text)

Max Activating Token Index: 7

Text #11

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3484. Min Act: -0.1700

Data Index: 7902423 (Open Web Text)

Max Activating Token Index: 170

Text #12

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3382. Min Act: -0.1700

Data Index: 7519295 (Open Web Text)

Max Activating Token Index: 636

Text #13

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3541. Min Act: -0.1700

Data Index: 1830019 (Open Web Text)

Max Activating Token Index: 992

Text #14

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3525. Min Act: -0.1700

Data Index: 4284741 (Open Web Text)

Max Activating Token Index: 698

Text #15

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3569. Min Act: -0.1700

Data Index: 4270594 (Open Web Text)

Max Activating Token Index: 917

Text #16

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3554. Min Act: -0.1700

Data Index: 3951452 (Open Web Text)

Max Activating Token Index: 755

Text #17

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3268. Min Act: -0.1700

Data Index: 7255765 (Open Web Text)

Max Activating Token Index: 953

Text #18

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3704. Min Act: -0.1700

Data Index: 5705251 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 4.5517. Min Range: -4.5517

Max Act: 4.3704. Min Act: -0.1700

Data Index: 5324768 (Open Web Text)

Max Activating Token Index: 98