Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2829 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0382. Min Range: -4.0382

Max Act: 4.0382. Min Act: -0.1700

Data Index: 6038814 (Open Web Text)

Max Activating Token Index: 33

Text #1

Max Range: 4.0382. Min Range: -4.0382

Max Act: 4.0164. Min Act: -0.1700

Data Index: 8008558 (Open Web Text)

Max Activating Token Index: 22

Text #2

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9716. Min Act: -0.1700

Data Index: 4400560 (Open Web Text)

Max Activating Token Index: 939

Text #3

Max Range: 4.0382. Min Range: -4.0382

Max Act: 4.0374. Min Act: -0.1700

Data Index: 3162481 (Open Web Text)

Max Activating Token Index: 20

Text #4

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9627. Min Act: -0.1700

Data Index: 5261719 (Open Web Text)

Max Activating Token Index: 39

Text #5

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9449. Min Act: -0.1700

Data Index: 6996497 (Open Web Text)

Max Activating Token Index: 359

Text #6

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9040. Min Act: -0.1700

Data Index: 5864527 (Open Web Text)

Max Activating Token Index: 994

Text #7

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9384. Min Act: -0.1700

Data Index: 1694962 (Open Web Text)

Max Activating Token Index: 419

Text #8

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.9311. Min Act: -0.1700

Data Index: 6110378 (Open Web Text)

Max Activating Token Index: 816

Text #9

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8875. Min Act: -0.1700

Data Index: 3278129 (Open Web Text)

Max Activating Token Index: 567

Text #10

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8978. Min Act: -0.1700

Data Index: 8604806 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8773. Min Act: -0.1700

Data Index: 1834137 (Open Web Text)

Max Activating Token Index: 529

Text #12

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8817. Min Act: -0.1700

Data Index: 4280409 (Open Web Text)

Max Activating Token Index: 636

Text #13

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8577. Min Act: -0.1700

Data Index: 8122684 (Open Web Text)

Max Activating Token Index: 918

Text #14

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8415. Min Act: -0.1700

Data Index: 6468759 (Open Web Text)

Max Activating Token Index: 853

Text #15

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8359. Min Act: -0.1700

Data Index: 406211 (Open Web Text)

Max Activating Token Index: 971

Text #16

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8531. Min Act: -0.1700

Data Index: 360453 (Open Web Text)

Max Activating Token Index: 67

Text #17

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.7802. Min Act: -0.1700

Data Index: 5965764 (Open Web Text)

Max Activating Token Index: 502

Text #18

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8068. Min Act: -0.1700

Data Index: 5210149 (Open Web Text)

Max Activating Token Index: 659

Text #19

Max Range: 4.0382. Min Range: -4.0382

Max Act: 3.8178. Min Act: -0.1700

Data Index: 496454 (Open Web Text)

Max Activating Token Index: 406