Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2686 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4891. Min Act: -0.1700

Data Index: 1869568 (Open Web Text)

Max Activating Token Index: 1017

Text #1

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4818. Min Act: -0.1700

Data Index: 8599890 (Open Web Text)

Max Activating Token Index: 316

Text #2

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4887. Min Act: -0.1700

Data Index: 4862110 (Open Web Text)

Max Activating Token Index: 838

Text #3

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4468. Min Act: -0.1700

Data Index: 5583291 (Open Web Text)

Max Activating Token Index: 398

Text #4

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4775. Min Act: -0.1700

Data Index: 5823513 (Open Web Text)

Max Activating Token Index: 623

Text #5

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4221. Min Act: -0.1700

Data Index: 3794222 (Open Web Text)

Max Activating Token Index: 606

Text #6

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4302. Min Act: -0.1700

Data Index: 6420511 (Open Web Text)

Max Activating Token Index: 418

Text #7

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4233. Min Act: -0.1700

Data Index: 7463426 (Open Web Text)

Max Activating Token Index: 664

Text #8

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4334. Min Act: -0.1700

Data Index: 1404349 (Open Web Text)

Max Activating Token Index: 852

Text #9

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4262. Min Act: -0.1700

Data Index: 2869539 (Open Web Text)

Max Activating Token Index: 357

Text #10

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4257. Min Act: -0.1700

Data Index: 7946007 (Open Web Text)

Max Activating Token Index: 238

Text #11

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4048. Min Act: -0.1700

Data Index: 7994715 (Open Web Text)

Max Activating Token Index: 276

Text #12

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.3909. Min Act: -0.1700

Data Index: 3910351 (Open Web Text)

Max Activating Token Index: 185

Text #13

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4028. Min Act: -0.1700

Data Index: 7331456 (Open Web Text)

Max Activating Token Index: 865

Text #14

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4023. Min Act: -0.1700

Data Index: 96812 (Open Web Text)

Max Activating Token Index: 218

Text #15

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4052. Min Act: -0.1700

Data Index: 4500859 (Open Web Text)

Max Activating Token Index: 430

Text #16

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.3798. Min Act: -0.1700

Data Index: 4079200 (Open Web Text)

Max Activating Token Index: 708

Text #17

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.4001. Min Act: -0.1700

Data Index: 2625566 (Open Web Text)

Max Activating Token Index: 776

Text #18

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.3802. Min Act: -0.1700

Data Index: 2763119 (Open Web Text)

Max Activating Token Index: 445

Text #19

Max Range: 3.4891. Min Range: -3.4891

Max Act: 3.3944. Min Act: -0.1700

Data Index: 6236313 (Open Web Text)

Max Activating Token Index: 791