Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2945 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gpt2-small')

Text #0

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.4411. Min Act: -0.1700

Data Index: 184520 (Open Web Text)

Max Activating Token Index: 309

Text #1

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.3198. Min Act: -0.1700

Data Index: 1244134 (Open Web Text)

Max Activating Token Index: 628

Text #2

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.1869. Min Act: -0.1700

Data Index: 406050 (Open Web Text)

Max Activating Token Index: 825

Text #3

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.2234. Min Act: -0.1700

Data Index: 184526 (Open Web Text)

Max Activating Token Index: 583

Text #4

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.1600. Min Act: -0.1700

Data Index: 7168358 (Open Web Text)

Max Activating Token Index: 611

Text #5

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.1167. Min Act: -0.1700

Data Index: 2070301 (Open Web Text)

Max Activating Token Index: 981

Text #6

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.1290. Min Act: -0.1700

Data Index: 2499321 (Open Web Text)

Max Activating Token Index: 83

Text #7

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.0932. Min Act: -0.1700

Data Index: 5408939 (Open Web Text)

Max Activating Token Index: 590

Text #8

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.0937. Min Act: -0.1700

Data Index: 3695955 (Open Web Text)

Max Activating Token Index: 379

Text #9

Max Range: 7.4411. Min Range: -7.4411

Max Act: 7.0513. Min Act: -0.1700

Data Index: 3161322 (Open Web Text)

Max Activating Token Index: 608

Text #10

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.9836. Min Act: -0.1700

Data Index: 3301099 (Open Web Text)

Max Activating Token Index: 508

Text #11

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.9698. Min Act: -0.1700

Data Index: 4882280 (Open Web Text)

Max Activating Token Index: 717

Text #12

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.9144. Min Act: -0.1700

Data Index: 2063420 (Open Web Text)

Max Activating Token Index: 134

Text #13

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8874. Min Act: -0.1700

Data Index: 7168357 (Open Web Text)

Max Activating Token Index: 987

Text #14

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8614. Min Act: -0.1700

Data Index: 503871 (Open Web Text)

Max Activating Token Index: 312

Text #15

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8628. Min Act: -0.1700

Data Index: 3880804 (Open Web Text)

Max Activating Token Index: 628

Text #16

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8503. Min Act: -0.1700

Data Index: 7588223 (Open Web Text)

Max Activating Token Index: 888

Text #17

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8355. Min Act: -0.1700

Data Index: 7921876 (Open Web Text)

Max Activating Token Index: 410

Text #18

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8253. Min Act: -0.1700

Data Index: 4017920 (Open Web Text)

Max Activating Token Index: 441

Text #19

Max Range: 7.4411. Min Range: -7.4411

Max Act: 6.8195. Min Act: -0.1700

Data Index: 6527979 (Open Web Text)

Max Activating Token Index: 276