Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1557 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.3175. Min Act: -0.1700

Data Index: 5735467 (Open Web Text)

Max Activating Token Index: 394

Text #1

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.3102. Min Act: -0.1700

Data Index: 7119054 (Open Web Text)

Max Activating Token Index: 620

Text #2

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2851. Min Act: -0.1700

Data Index: 1294224 (Open Web Text)

Max Activating Token Index: 1021

Text #3

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2981. Min Act: -0.1700

Data Index: 3376399 (Open Web Text)

Max Activating Token Index: 16

Text #4

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2566. Min Act: -0.1700

Data Index: 6178947 (Open Web Text)

Max Activating Token Index: 275

Text #5

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2779. Min Act: -0.1700

Data Index: 593039 (Open Web Text)

Max Activating Token Index: 259

Text #6

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2739. Min Act: -0.1700

Data Index: 4166599 (Open Web Text)

Max Activating Token Index: 816

Text #7

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2625. Min Act: -0.1700

Data Index: 5136351 (Open Web Text)

Max Activating Token Index: 445

Text #8

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2411. Min Act: -0.1700

Data Index: 878693 (Open Web Text)

Max Activating Token Index: 940

Text #9

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2551. Min Act: -0.1700

Data Index: 1145734 (Open Web Text)

Max Activating Token Index: 579

Text #10

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2481. Min Act: -0.1700

Data Index: 7179487 (Open Web Text)

Max Activating Token Index: 894

Text #11

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2208. Min Act: -0.1700

Data Index: 3301123 (Open Web Text)

Max Activating Token Index: 164

Text #12

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.1804. Min Act: -0.1700

Data Index: 5962031 (Open Web Text)

Max Activating Token Index: 745

Text #13

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2175. Min Act: -0.1700

Data Index: 7222770 (Open Web Text)

Max Activating Token Index: 481

Text #14

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.1927. Min Act: -0.1700

Data Index: 3962009 (Open Web Text)

Max Activating Token Index: 880

Text #15

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2396. Min Act: -0.1700

Data Index: 6948147 (Open Web Text)

Max Activating Token Index: 275

Text #16

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2014. Min Act: -0.1700

Data Index: 6133922 (Open Web Text)

Max Activating Token Index: 375

Text #17

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2073. Min Act: -0.1700

Data Index: 4519815 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.1715. Min Act: -0.1700

Data Index: 4862491 (Open Web Text)

Max Activating Token Index: 610

Text #19

Max Range: 3.3175. Min Range: -3.3175

Max Act: 3.2081. Min Act: -0.1700

Data Index: 5531562 (Open Web Text)

Max Activating Token Index: 657