Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1025 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.8005. Min Act: -0.1700

Data Index: 3795826 (Open Web Text)

Max Activating Token Index: 406

Text #1

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7539. Min Act: -0.1700

Data Index: 3537237 (Open Web Text)

Max Activating Token Index: 994

Text #2

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7604. Min Act: -0.1700

Data Index: 1111022 (Open Web Text)

Max Activating Token Index: 1019

Text #3

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7393. Min Act: -0.1700

Data Index: 978777 (Open Web Text)

Max Activating Token Index: 485

Text #4

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7011. Min Act: -0.1700

Data Index: 4275480 (Open Web Text)

Max Activating Token Index: 702

Text #5

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7326. Min Act: -0.1700

Data Index: 7771698 (Open Web Text)

Max Activating Token Index: 202

Text #6

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6815. Min Act: -0.1700

Data Index: 2171883 (Open Web Text)

Max Activating Token Index: 436

Text #7

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.7095. Min Act: -0.1700

Data Index: 1004287 (Open Web Text)

Max Activating Token Index: 152

Text #8

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6453. Min Act: -0.1700

Data Index: 3763413 (Open Web Text)

Max Activating Token Index: 531

Text #9

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6487. Min Act: -0.1700

Data Index: 49949 (Open Web Text)

Max Activating Token Index: 917

Text #10

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6266. Min Act: -0.1700

Data Index: 1755215 (Open Web Text)

Max Activating Token Index: 194

Text #11

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6530. Min Act: -0.1700

Data Index: 2614627 (Open Web Text)

Max Activating Token Index: 120

Text #12

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6273. Min Act: -0.1700

Data Index: 7197010 (Open Web Text)

Max Activating Token Index: 859

Text #13

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6308. Min Act: -0.1700

Data Index: 6946612 (Open Web Text)

Max Activating Token Index: 795

Text #14

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6197. Min Act: -0.1700

Data Index: 569652 (Open Web Text)

Max Activating Token Index: 147

Text #15

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.5780. Min Act: -0.1700

Data Index: 5412361 (Open Web Text)

Max Activating Token Index: 584

Text #16

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6003. Min Act: -0.1700

Data Index: 7785006 (Open Web Text)

Max Activating Token Index: 362

Text #17

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.6006. Min Act: -0.1700

Data Index: 2369076 (Open Web Text)

Max Activating Token Index: 607

Text #18

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.5856. Min Act: -0.1700

Data Index: 3756285 (Open Web Text)

Max Activating Token Index: 730

Text #19

Max Range: 4.8005. Min Range: -4.8005

Max Act: 4.5579. Min Act: -0.1700

Data Index: 2020784 (Open Web Text)

Max Activating Token Index: 62