Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1643 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.8898. Min Act: -0.0177

Data Index: 1042043 (Open Web Text)

Max Activating Token Index: 389

Text #1

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.8462. Min Act: -0.0989

Data Index: 8628515 (Open Web Text)

Max Activating Token Index: 992

Text #2

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.8336. Min Act: -0.0177

Data Index: 1195687 (Open Web Text)

Max Activating Token Index: 787

Text #3

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.8206. Min Act: -0.0177

Data Index: 432050 (Open Web Text)

Max Activating Token Index: 534

Text #4

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7954. Min Act: -0.0177

Data Index: 3813710 (Open Web Text)

Max Activating Token Index: 878

Text #5

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7797. Min Act: -0.0177

Data Index: 8354945 (Open Web Text)

Max Activating Token Index: 210

Text #6

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7694. Min Act: -0.0218

Data Index: 4076004 (Open Web Text)

Max Activating Token Index: 579

Text #7

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7704. Min Act: -0.0345

Data Index: 432040 (Open Web Text)

Max Activating Token Index: 456

Text #8

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7654. Min Act: -0.0640

Data Index: 4284922 (Open Web Text)

Max Activating Token Index: 352

Text #9

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7647. Min Act: -0.0374

Data Index: 5004717 (Open Web Text)

Max Activating Token Index: 477

Text #10

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7529. Min Act: -0.0177

Data Index: 197239 (Open Web Text)

Max Activating Token Index: 519

Text #11

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7551. Min Act: -0.0883

Data Index: 544469 (Open Web Text)

Max Activating Token Index: 387

Text #12

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7492. Min Act: -0.0656

Data Index: 8722266 (Open Web Text)

Max Activating Token Index: 161

Text #13

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7548. Min Act: -0.0359

Data Index: 1308525 (Open Web Text)

Max Activating Token Index: 489

Text #14

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7530. Min Act: -0.0177

Data Index: 4284921 (Open Web Text)

Max Activating Token Index: 355

Text #15

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7542. Min Act: -0.0177

Data Index: 3344604 (Open Web Text)

Max Activating Token Index: 539

Text #16

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7366. Min Act: -0.0608

Data Index: 2796453 (Open Web Text)

Max Activating Token Index: 722

Text #17

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7433. Min Act: -0.0547

Data Index: 672640 (Open Web Text)

Max Activating Token Index: 643

Text #18

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7453. Min Act: -0.0177

Data Index: 1897259 (Open Web Text)

Max Activating Token Index: 696

Text #19

Max Range: 1.8898. Min Range: -1.8898

Max Act: 1.7442. Min Act: -0.0222

Data Index: 2895670 (Open Web Text)

Max Activating Token Index: 409