Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1471 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.8885. Min Act: -0.1700

Data Index: 5624750 (Open Web Text)

Max Activating Token Index: 924

Text #1

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.8910. Min Act: -0.1700

Data Index: 8039838 (Open Web Text)

Max Activating Token Index: 431

Text #2

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.8871. Min Act: -0.1700

Data Index: 4352726 (Open Web Text)

Max Activating Token Index: 767

Text #3

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.8332. Min Act: -0.1700

Data Index: 2708145 (Open Web Text)

Max Activating Token Index: 426

Text #4

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.8131. Min Act: -0.1700

Data Index: 6671654 (Open Web Text)

Max Activating Token Index: 364

Text #5

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7836. Min Act: -0.1700

Data Index: 6039603 (Open Web Text)

Max Activating Token Index: 94

Text #6

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7785. Min Act: -0.1700

Data Index: 3713236 (Open Web Text)

Max Activating Token Index: 110

Text #7

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7277. Min Act: -0.1700

Data Index: 6733206 (Open Web Text)

Max Activating Token Index: 883

Text #8

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7414. Min Act: -0.1700

Data Index: 6630978 (Open Web Text)

Max Activating Token Index: 992

Text #9

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7356. Min Act: -0.1700

Data Index: 4471874 (Open Web Text)

Max Activating Token Index: 286

Text #10

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7373. Min Act: -0.1700

Data Index: 5892117 (Open Web Text)

Max Activating Token Index: 576

Text #11

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7031. Min Act: -0.1700

Data Index: 5454227 (Open Web Text)

Max Activating Token Index: 467

Text #12

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7329. Min Act: -0.1700

Data Index: 3782885 (Open Web Text)

Max Activating Token Index: 592

Text #13

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.7316. Min Act: -0.1700

Data Index: 7955627 (Open Web Text)

Max Activating Token Index: 836

Text #14

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6829. Min Act: -0.1700

Data Index: 7677876 (Open Web Text)

Max Activating Token Index: 388

Text #15

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6831. Min Act: -0.1700

Data Index: 1844598 (Open Web Text)

Max Activating Token Index: 459

Text #16

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6778. Min Act: -0.1700

Data Index: 8543707 (Open Web Text)

Max Activating Token Index: 924

Text #17

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6776. Min Act: -0.1700

Data Index: 352468 (Open Web Text)

Max Activating Token Index: 814

Text #18

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6506. Min Act: -0.1700

Data Index: 4776703 (Open Web Text)

Max Activating Token Index: 413

Text #19

Max Range: 3.8910. Min Range: -3.8910

Max Act: 3.6793. Min Act: -0.1700

Data Index: 4090876 (Open Web Text)

Max Activating Token Index: 505