Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1951 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.7746. Min Act: -0.1700

Data Index: 4097765 (Open Web Text)

Max Activating Token Index: 147

Text #1

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.7120. Min Act: -0.1700

Data Index: 2070564 (Open Web Text)

Max Activating Token Index: 692

Text #2

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6717. Min Act: -0.1700

Data Index: 2581321 (Open Web Text)

Max Activating Token Index: 288

Text #3

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6152. Min Act: -0.1700

Data Index: 7026735 (Open Web Text)

Max Activating Token Index: 558

Text #4

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6294. Min Act: -0.1700

Data Index: 1144766 (Open Web Text)

Max Activating Token Index: 106

Text #5

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5919. Min Act: -0.1700

Data Index: 7118014 (Open Web Text)

Max Activating Token Index: 610

Text #6

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6232. Min Act: -0.1700

Data Index: 5326084 (Open Web Text)

Max Activating Token Index: 544

Text #7

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5610. Min Act: -0.1700

Data Index: 6076487 (Open Web Text)

Max Activating Token Index: 366

Text #8

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6072. Min Act: -0.1700

Data Index: 439862 (Open Web Text)

Max Activating Token Index: 858

Text #9

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6045. Min Act: -0.1700

Data Index: 3250898 (Open Web Text)

Max Activating Token Index: 106

Text #10

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5877. Min Act: -0.1700

Data Index: 4534163 (Open Web Text)

Max Activating Token Index: 299

Text #11

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.6019. Min Act: -0.1700

Data Index: 633854 (Open Web Text)

Max Activating Token Index: 678

Text #12

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5930. Min Act: -0.1700

Data Index: 4686941 (Open Web Text)

Max Activating Token Index: 334

Text #13

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5803. Min Act: -0.1700

Data Index: 3927599 (Open Web Text)

Max Activating Token Index: 403

Text #14

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5598. Min Act: -0.1700

Data Index: 3726501 (Open Web Text)

Max Activating Token Index: 39

Text #15

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5737. Min Act: -0.1700

Data Index: 4116428 (Open Web Text)

Max Activating Token Index: 589

Text #16

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5560. Min Act: -0.1700

Data Index: 5589507 (Open Web Text)

Max Activating Token Index: 214

Text #17

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5478. Min Act: -0.1700

Data Index: 1985837 (Open Web Text)

Max Activating Token Index: 410

Text #18

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5959. Min Act: -0.1700

Data Index: 5733735 (Open Web Text)

Max Activating Token Index: 753

Text #19

Max Range: 4.7746. Min Range: -4.7746

Max Act: 4.5647. Min Act: -0.1700

Data Index: 4860243 (Open Web Text)

Max Activating Token Index: 670