Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2853 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.9470. Min Act: -0.1700

Data Index: 2978220 (Open Web Text)

Max Activating Token Index: 291

Text #1

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.6674. Min Act: -0.1700

Data Index: 8577206 (Open Web Text)

Max Activating Token Index: 200

Text #2

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.6893. Min Act: -0.1700

Data Index: 1507796 (Open Web Text)

Max Activating Token Index: 232

Text #3

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.6961. Min Act: -0.1700

Data Index: 594791 (Open Web Text)

Max Activating Token Index: 707

Text #4

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.6428. Min Act: -0.1700

Data Index: 1105707 (Open Web Text)

Max Activating Token Index: 745

Text #5

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.5424. Min Act: -0.1700

Data Index: 8558887 (Open Web Text)

Max Activating Token Index: 256

Text #6

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.5987. Min Act: -0.1700

Data Index: 7689809 (Open Web Text)

Max Activating Token Index: 144

Text #7

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.5670. Min Act: -0.1700

Data Index: 8348200 (Open Web Text)

Max Activating Token Index: 119

Text #8

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.5708. Min Act: -0.1700

Data Index: 6522794 (Open Web Text)

Max Activating Token Index: 312

Text #9

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4818. Min Act: -0.1700

Data Index: 7137435 (Open Web Text)

Max Activating Token Index: 745

Text #10

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4917. Min Act: -0.1700

Data Index: 8483862 (Open Web Text)

Max Activating Token Index: 143

Text #11

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4570. Min Act: -0.1700

Data Index: 7137437 (Open Web Text)

Max Activating Token Index: 377

Text #12

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4680. Min Act: -0.1700

Data Index: 7973998 (Open Web Text)

Max Activating Token Index: 93

Text #13

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4656. Min Act: -0.1700

Data Index: 494522 (Open Web Text)

Max Activating Token Index: 188

Text #14

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4380. Min Act: -0.1700

Data Index: 4355424 (Open Web Text)

Max Activating Token Index: 535

Text #15

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4276. Min Act: -0.1700

Data Index: 3314157 (Open Web Text)

Max Activating Token Index: 190

Text #16

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.4220. Min Act: -0.1700

Data Index: 7952272 (Open Web Text)

Max Activating Token Index: 968

Text #17

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.3854. Min Act: -0.1700

Data Index: 4661922 (Open Web Text)

Max Activating Token Index: 31

Text #18

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.3455. Min Act: -0.1700

Data Index: 2298875 (Open Web Text)

Max Activating Token Index: 83

Text #19

Max Range: 4.9470. Min Range: -4.9470

Max Act: 4.3431. Min Act: -0.1700

Data Index: 594790 (Open Web Text)

Max Activating Token Index: 891