Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2973 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4795. Min Act: -0.1700

Data Index: 1939968 (Open Web Text)

Max Activating Token Index: 293

Text #1

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4786. Min Act: -0.1700

Data Index: 4147560 (Open Web Text)

Max Activating Token Index: 250

Text #2

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.5291. Min Act: -0.1700

Data Index: 2751342 (Open Web Text)

Max Activating Token Index: 890

Text #3

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4352. Min Act: -0.1700

Data Index: 104351 (Open Web Text)

Max Activating Token Index: 103

Text #4

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4391. Min Act: -0.1700

Data Index: 3446738 (Open Web Text)

Max Activating Token Index: 677

Text #5

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4211. Min Act: -0.1700

Data Index: 6482956 (Open Web Text)

Max Activating Token Index: 524

Text #6

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3854. Min Act: -0.1700

Data Index: 2897608 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.4049. Min Act: -0.1700

Data Index: 7380000 (Open Web Text)

Max Activating Token Index: 877

Text #8

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3991. Min Act: -0.1700

Data Index: 7738214 (Open Web Text)

Max Activating Token Index: 111

Text #9

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3907. Min Act: -0.1700

Data Index: 6796573 (Open Web Text)

Max Activating Token Index: 463

Text #10

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3385. Min Act: -0.1700

Data Index: 6626587 (Open Web Text)

Max Activating Token Index: 132

Text #11

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3172. Min Act: -0.1700

Data Index: 4749386 (Open Web Text)

Max Activating Token Index: 589

Text #12

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3631. Min Act: -0.1700

Data Index: 3048817 (Open Web Text)

Max Activating Token Index: 416

Text #13

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3487. Min Act: -0.1700

Data Index: 6407379 (Open Web Text)

Max Activating Token Index: 574

Text #14

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3409. Min Act: -0.1700

Data Index: 5929077 (Open Web Text)

Max Activating Token Index: 978

Text #15

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3312. Min Act: -0.1700

Data Index: 3671425 (Open Web Text)

Max Activating Token Index: 786

Text #16

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3352. Min Act: -0.1700

Data Index: 6542931 (Open Web Text)

Max Activating Token Index: 755

Text #17

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3158. Min Act: -0.1700

Data Index: 4853783 (Open Web Text)

Max Activating Token Index: 826

Text #18

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3263. Min Act: -0.1700

Data Index: 5135586 (Open Web Text)

Max Activating Token Index: 140

Text #19

Max Range: 3.5291. Min Range: -3.5291

Max Act: 3.3275. Min Act: -0.1700

Data Index: 333059 (Open Web Text)

Max Activating Token Index: 168