Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2890 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.7231. Min Act: -0.1700

Data Index: 7904247 (Open Web Text)

Max Activating Token Index: 384

Text #1

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.7615. Min Act: -0.1700

Data Index: 3559536 (Open Web Text)

Max Activating Token Index: 990

Text #2

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.6535. Min Act: -0.1700

Data Index: 5127518 (Open Web Text)

Max Activating Token Index: 482

Text #3

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5470. Min Act: -0.1700

Data Index: 4573266 (Open Web Text)

Max Activating Token Index: 294

Text #4

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.6122. Min Act: -0.1700

Data Index: 1003587 (Open Web Text)

Max Activating Token Index: 432

Text #5

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5962. Min Act: -0.1700

Data Index: 856242 (Open Web Text)

Max Activating Token Index: 260

Text #6

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5690. Min Act: -0.1700

Data Index: 3891601 (Open Web Text)

Max Activating Token Index: 208

Text #7

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5220. Min Act: -0.1700

Data Index: 4054010 (Open Web Text)

Max Activating Token Index: 262

Text #8

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.6018. Min Act: -0.1700

Data Index: 5296508 (Open Web Text)

Max Activating Token Index: 458

Text #9

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5602. Min Act: -0.1700

Data Index: 7535440 (Open Web Text)

Max Activating Token Index: 625

Text #10

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4526. Min Act: -0.1700

Data Index: 6018054 (Open Web Text)

Max Activating Token Index: 212

Text #11

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5520. Min Act: -0.1700

Data Index: 7710935 (Open Web Text)

Max Activating Token Index: 410

Text #12

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4874. Min Act: -0.1700

Data Index: 5683663 (Open Web Text)

Max Activating Token Index: 350

Text #13

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4826. Min Act: -0.1700

Data Index: 4461488 (Open Web Text)

Max Activating Token Index: 86

Text #14

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.5263. Min Act: -0.1700

Data Index: 4477334 (Open Web Text)

Max Activating Token Index: 902

Text #15

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4571. Min Act: -0.1700

Data Index: 4494553 (Open Web Text)

Max Activating Token Index: 806

Text #16

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4867. Min Act: -0.1700

Data Index: 6018056 (Open Web Text)

Max Activating Token Index: 433

Text #17

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4916. Min Act: -0.1700

Data Index: 1663017 (Open Web Text)

Max Activating Token Index: 373

Text #18

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4563. Min Act: -0.1700

Data Index: 6314032 (Open Web Text)

Max Activating Token Index: 31

Text #19

Max Range: 4.7615. Min Range: -4.7615

Max Act: 4.4467. Min Act: -0.1700

Data Index: 7297270 (Open Web Text)

Max Activating Token Index: 358