Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1032 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8833. Min Act: -0.1700

Data Index: 535702 (Open Web Text)

Max Activating Token Index: 455

Text #1

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8726. Min Act: -0.1700

Data Index: 5340777 (Open Web Text)

Max Activating Token Index: 265

Text #2

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8295. Min Act: -0.1700

Data Index: 4134177 (Open Web Text)

Max Activating Token Index: 163

Text #3

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8838. Min Act: -0.1700

Data Index: 4668497 (Open Web Text)

Max Activating Token Index: 861

Text #4

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8328. Min Act: -0.1700

Data Index: 6014034 (Open Web Text)

Max Activating Token Index: 697

Text #5

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8367. Min Act: -0.1700

Data Index: 6221720 (Open Web Text)

Max Activating Token Index: 318

Text #6

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8010. Min Act: -0.1700

Data Index: 5340778 (Open Web Text)

Max Activating Token Index: 170

Text #7

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7701. Min Act: -0.1700

Data Index: 7551577 (Open Web Text)

Max Activating Token Index: 388

Text #8

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.8060. Min Act: -0.1700

Data Index: 2209400 (Open Web Text)

Max Activating Token Index: 49

Text #9

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7255. Min Act: -0.1700

Data Index: 6695274 (Open Web Text)

Max Activating Token Index: 148

Text #10

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7180. Min Act: -0.1700

Data Index: 7884137 (Open Web Text)

Max Activating Token Index: 214

Text #11

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7567. Min Act: -0.1700

Data Index: 3265038 (Open Web Text)

Max Activating Token Index: 312

Text #12

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7494. Min Act: -0.1700

Data Index: 6575271 (Open Web Text)

Max Activating Token Index: 730

Text #13

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7973. Min Act: -0.1700

Data Index: 7049032 (Open Web Text)

Max Activating Token Index: 648

Text #14

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7772. Min Act: -0.1700

Data Index: 2049507 (Open Web Text)

Max Activating Token Index: 493

Text #15

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7695. Min Act: -0.1700

Data Index: 3409854 (Open Web Text)

Max Activating Token Index: 439

Text #16

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.7061. Min Act: -0.1700

Data Index: 468693 (Open Web Text)

Max Activating Token Index: 629

Text #17

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6603. Min Act: -0.1700

Data Index: 2638596 (Open Web Text)

Max Activating Token Index: 921

Text #18

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6778. Min Act: -0.1700

Data Index: 5026756 (Open Web Text)

Max Activating Token Index: 139

Text #19

Max Range: 3.8838. Min Range: -3.8838

Max Act: 3.6246. Min Act: -0.1700

Data Index: 6189558 (Open Web Text)

Max Activating Token Index: 550