Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 133 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.8087. Min Act: -0.1689

Data Index: 6738693 (Open Web Text)

Max Activating Token Index: 223

Text #1

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7903. Min Act: -0.1700

Data Index: 2192176 (Open Web Text)

Max Activating Token Index: 334

Text #2

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7785. Min Act: -0.1700

Data Index: 6509024 (Open Web Text)

Max Activating Token Index: 82

Text #3

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7750. Min Act: -0.1700

Data Index: 4397505 (Open Web Text)

Max Activating Token Index: 536

Text #4

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7605. Min Act: -0.1700

Data Index: 6543718 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7582. Min Act: -0.1700

Data Index: 4157371 (Open Web Text)

Max Activating Token Index: 440

Text #6

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7560. Min Act: -0.1700

Data Index: 5246972 (Open Web Text)

Max Activating Token Index: 586

Text #7

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7538. Min Act: -0.1700

Data Index: 6509029 (Open Web Text)

Max Activating Token Index: 520

Text #8

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7445. Min Act: -0.1699

Data Index: 516824 (Open Web Text)

Max Activating Token Index: 548

Text #9

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7533. Min Act: -0.1700

Data Index: 7009382 (Open Web Text)

Max Activating Token Index: 698

Text #10

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7423. Min Act: -0.1700

Data Index: 2654882 (Open Web Text)

Max Activating Token Index: 536

Text #11

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7391. Min Act: -0.1700

Data Index: 1587332 (Open Web Text)

Max Activating Token Index: 560

Text #12

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7391. Min Act: -0.1700

Data Index: 4684207 (Open Web Text)

Max Activating Token Index: 903

Text #13

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7377. Min Act: -0.1700

Data Index: 2075585 (Open Web Text)

Max Activating Token Index: 407

Text #14

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7326. Min Act: -0.1699

Data Index: 6509021 (Open Web Text)

Max Activating Token Index: 525

Text #15

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7375. Min Act: -0.1700

Data Index: 7050178 (Open Web Text)

Max Activating Token Index: 949

Text #16

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7370. Min Act: -0.1700

Data Index: 5870596 (Open Web Text)

Max Activating Token Index: 497

Text #17

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7275. Min Act: -0.1700

Data Index: 2012692 (Open Web Text)

Max Activating Token Index: 626

Text #18

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7327. Min Act: -0.1698

Data Index: 334772 (Open Web Text)

Max Activating Token Index: 213

Text #19

Max Range: 1.8087. Min Range: -1.8087

Max Act: 1.7290. Min Act: -0.1700

Data Index: 3662431 (Open Web Text)

Max Activating Token Index: 736