Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1098 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.9555. Min Act: -0.1700

Data Index: 7174446 (Open Web Text)

Max Activating Token Index: 522

Text #1

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.9429. Min Act: -0.1700

Data Index: 7149924 (Open Web Text)

Max Activating Token Index: 21

Text #2

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8766. Min Act: -0.1700

Data Index: 3652104 (Open Web Text)

Max Activating Token Index: 402

Text #3

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8861. Min Act: -0.1700

Data Index: 4784819 (Open Web Text)

Max Activating Token Index: 249

Text #4

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.9116. Min Act: -0.1700

Data Index: 3123909 (Open Web Text)

Max Activating Token Index: 180

Text #5

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8486. Min Act: -0.1700

Data Index: 6937418 (Open Web Text)

Max Activating Token Index: 405

Text #6

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8585. Min Act: -0.1700

Data Index: 7659029 (Open Web Text)

Max Activating Token Index: 183

Text #7

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8629. Min Act: -0.1700

Data Index: 8664511 (Open Web Text)

Max Activating Token Index: 854

Text #8

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8513. Min Act: -0.1700

Data Index: 3752204 (Open Web Text)

Max Activating Token Index: 719

Text #9

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7731. Min Act: -0.1700

Data Index: 6166041 (Open Web Text)

Max Activating Token Index: 76

Text #10

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8147. Min Act: -0.1700

Data Index: 6339323 (Open Web Text)

Max Activating Token Index: 22

Text #11

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8623. Min Act: -0.1700

Data Index: 104710 (Open Web Text)

Max Activating Token Index: 992

Text #12

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7967. Min Act: -0.1700

Data Index: 2228787 (Open Web Text)

Max Activating Token Index: 675

Text #13

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8485. Min Act: -0.1700

Data Index: 3893674 (Open Web Text)

Max Activating Token Index: 984

Text #14

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7794. Min Act: -0.1700

Data Index: 2432540 (Open Web Text)

Max Activating Token Index: 71

Text #15

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7459. Min Act: -0.1700

Data Index: 7632112 (Open Web Text)

Max Activating Token Index: 586

Text #16

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8113. Min Act: -0.1700

Data Index: 5511555 (Open Web Text)

Max Activating Token Index: 749

Text #17

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.8467. Min Act: -0.1700

Data Index: 4658121 (Open Web Text)

Max Activating Token Index: 934

Text #18

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7620. Min Act: -0.1700

Data Index: 8375695 (Open Web Text)

Max Activating Token Index: 157

Text #19

Max Range: 4.9555. Min Range: -4.9555

Max Act: 4.7391. Min Act: -0.1700

Data Index: 7158330 (Open Web Text)

Max Activating Token Index: 488