Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2379 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.6144. Min Act: -0.1700

Data Index: 4118304 (Open Web Text)

Max Activating Token Index: 205

Text #1

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.5383. Min Act: -0.1700

Data Index: 3321860 (Open Web Text)

Max Activating Token Index: 275

Text #2

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.4124. Min Act: -0.1700

Data Index: 243557 (Open Web Text)

Max Activating Token Index: 342

Text #3

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3506. Min Act: -0.1700

Data Index: 4302152 (Open Web Text)

Max Activating Token Index: 86

Text #4

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3453. Min Act: -0.1700

Data Index: 3418556 (Open Web Text)

Max Activating Token Index: 801

Text #5

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3392. Min Act: -0.1700

Data Index: 7300879 (Open Web Text)

Max Activating Token Index: 694

Text #6

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3075. Min Act: -0.1700

Data Index: 6798187 (Open Web Text)

Max Activating Token Index: 84

Text #7

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3219. Min Act: -0.1700

Data Index: 1196713 (Open Web Text)

Max Activating Token Index: 557

Text #8

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3203. Min Act: -0.1700

Data Index: 8550425 (Open Web Text)

Max Activating Token Index: 144

Text #9

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3572. Min Act: -0.1700

Data Index: 4975328 (Open Web Text)

Max Activating Token Index: 758

Text #10

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3027. Min Act: -0.1700

Data Index: 6772022 (Open Web Text)

Max Activating Token Index: 428

Text #11

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2463. Min Act: -0.1700

Data Index: 4291513 (Open Web Text)

Max Activating Token Index: 267

Text #12

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2885. Min Act: -0.1700

Data Index: 6853326 (Open Web Text)

Max Activating Token Index: 144

Text #13

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3152. Min Act: -0.1700

Data Index: 945234 (Open Web Text)

Max Activating Token Index: 480

Text #14

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2876. Min Act: -0.1700

Data Index: 8148363 (Open Web Text)

Max Activating Token Index: 356

Text #15

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2727. Min Act: -0.1700

Data Index: 2814794 (Open Web Text)

Max Activating Token Index: 792

Text #16

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2871. Min Act: -0.1700

Data Index: 3229343 (Open Web Text)

Max Activating Token Index: 596

Text #17

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.3144. Min Act: -0.1700

Data Index: 5620073 (Open Web Text)

Max Activating Token Index: 586

Text #18

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2621. Min Act: -0.1700

Data Index: 450122 (Open Web Text)

Max Activating Token Index: 395

Text #19

Max Range: 4.6144. Min Range: -4.6144

Max Act: 4.2517. Min Act: -0.1700

Data Index: 7188113 (Open Web Text)

Max Activating Token Index: 505