Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1010 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.8241. Min Act: -0.1700

Data Index: 1870822 (Open Web Text)

Max Activating Token Index: 568

Text #1

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6997. Min Act: -0.1700

Data Index: 2636449 (Open Web Text)

Max Activating Token Index: 81

Text #2

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6961. Min Act: -0.1700

Data Index: 5265997 (Open Web Text)

Max Activating Token Index: 106

Text #3

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6323. Min Act: -0.1700

Data Index: 3073455 (Open Web Text)

Max Activating Token Index: 421

Text #4

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6096. Min Act: -0.1700

Data Index: 2915562 (Open Web Text)

Max Activating Token Index: 156

Text #5

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6425. Min Act: -0.1700

Data Index: 3862998 (Open Web Text)

Max Activating Token Index: 371

Text #6

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6128. Min Act: -0.1700

Data Index: 5105600 (Open Web Text)

Max Activating Token Index: 103

Text #7

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6131. Min Act: -0.1700

Data Index: 8001559 (Open Web Text)

Max Activating Token Index: 665

Text #8

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6266. Min Act: -0.1700

Data Index: 7854390 (Open Web Text)

Max Activating Token Index: 667

Text #9

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6067. Min Act: -0.1700

Data Index: 7067050 (Open Web Text)

Max Activating Token Index: 181

Text #10

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6238. Min Act: -0.1700

Data Index: 3766150 (Open Web Text)

Max Activating Token Index: 385

Text #11

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6348. Min Act: -0.1700

Data Index: 5523908 (Open Web Text)

Max Activating Token Index: 101

Text #12

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.6049. Min Act: -0.1700

Data Index: 3487791 (Open Web Text)

Max Activating Token Index: 901

Text #13

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5763. Min Act: -0.1700

Data Index: 1899754 (Open Web Text)

Max Activating Token Index: 792

Text #14

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5922. Min Act: -0.1700

Data Index: 860678 (Open Web Text)

Max Activating Token Index: 27

Text #15

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5766. Min Act: -0.1700

Data Index: 6730378 (Open Web Text)

Max Activating Token Index: 161

Text #16

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5373. Min Act: -0.1700

Data Index: 8400303 (Open Web Text)

Max Activating Token Index: 501

Text #17

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5734. Min Act: -0.1700

Data Index: 72103 (Open Web Text)

Max Activating Token Index: 894

Text #18

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5732. Min Act: -0.1700

Data Index: 3043749 (Open Web Text)

Max Activating Token Index: 557

Text #19

Max Range: 3.8241. Min Range: -3.8241

Max Act: 3.5840. Min Act: -0.1700

Data Index: 524913 (Open Web Text)

Max Activating Token Index: 727