Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2855 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.8280. Min Act: -0.1700

Data Index: 7160606 (Open Web Text)

Max Activating Token Index: 895

Text #1

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.7039. Min Act: -0.1700

Data Index: 5108349 (Open Web Text)

Max Activating Token Index: 641

Text #2

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.6995. Min Act: -0.1700

Data Index: 5840294 (Open Web Text)

Max Activating Token Index: 256

Text #3

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.5600. Min Act: -0.1700

Data Index: 8445412 (Open Web Text)

Max Activating Token Index: 912

Text #4

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.5014. Min Act: -0.1700

Data Index: 5579624 (Open Web Text)

Max Activating Token Index: 858

Text #5

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.5231. Min Act: -0.1700

Data Index: 4597794 (Open Web Text)

Max Activating Token Index: 865

Text #6

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.5016. Min Act: -0.1700

Data Index: 7708149 (Open Web Text)

Max Activating Token Index: 562

Text #7

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.4770. Min Act: -0.1700

Data Index: 7562768 (Open Web Text)

Max Activating Token Index: 330

Text #8

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.4794. Min Act: -0.1700

Data Index: 5749662 (Open Web Text)

Max Activating Token Index: 267

Text #9

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.4647. Min Act: -0.1700

Data Index: 414847 (Open Web Text)

Max Activating Token Index: 385

Text #10

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.4335. Min Act: -0.1700

Data Index: 6543395 (Open Web Text)

Max Activating Token Index: 346

Text #11

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.4144. Min Act: -0.1700

Data Index: 7893209 (Open Web Text)

Max Activating Token Index: 519

Text #12

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3968. Min Act: -0.1700

Data Index: 5403091 (Open Web Text)

Max Activating Token Index: 38

Text #13

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3723. Min Act: -0.1700

Data Index: 7110244 (Open Web Text)

Max Activating Token Index: 979

Text #14

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3466. Min Act: -0.1700

Data Index: 2941034 (Open Web Text)

Max Activating Token Index: 688

Text #15

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3811. Min Act: -0.1700

Data Index: 3003294 (Open Web Text)

Max Activating Token Index: 791

Text #16

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3643. Min Act: -0.1700

Data Index: 958473 (Open Web Text)

Max Activating Token Index: 609

Text #17

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.3688. Min Act: -0.1700

Data Index: 3262924 (Open Web Text)

Max Activating Token Index: 955

Text #18

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.2844. Min Act: -0.1700

Data Index: 4235539 (Open Web Text)

Max Activating Token Index: 428

Text #19

Max Range: 4.8280. Min Range: -4.8280

Max Act: 4.2878. Min Act: -0.1700

Data Index: 4395258 (Open Web Text)

Max Activating Token Index: 674