Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 948 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0382. Min Range: -3.0382

Max Act: 3.0382. Min Act: -0.1700

Data Index: 8152683 (Open Web Text)

Max Activating Token Index: 924

Text #1

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.9395. Min Act: -0.1700

Data Index: 5933292 (Open Web Text)

Max Activating Token Index: 762

Text #2

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.9080. Min Act: -0.1700

Data Index: 45067 (Open Web Text)

Max Activating Token Index: 686

Text #3

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7825. Min Act: -0.1700

Data Index: 3901730 (Open Web Text)

Max Activating Token Index: 681

Text #4

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7666. Min Act: -0.1700

Data Index: 8817818 (Open Web Text)

Max Activating Token Index: 365

Text #5

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7915. Min Act: -0.1700

Data Index: 7778028 (Open Web Text)

Max Activating Token Index: 768

Text #6

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7777. Min Act: -0.1700

Data Index: 2395735 (Open Web Text)

Max Activating Token Index: 733

Text #7

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7369. Min Act: -0.1700

Data Index: 6834686 (Open Web Text)

Max Activating Token Index: 658

Text #8

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7908. Min Act: -0.1700

Data Index: 3429240 (Open Web Text)

Max Activating Token Index: 142

Text #9

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7738. Min Act: -0.1700

Data Index: 2934127 (Open Web Text)

Max Activating Token Index: 350

Text #10

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7524. Min Act: -0.1700

Data Index: 2267276 (Open Web Text)

Max Activating Token Index: 512

Text #11

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7480. Min Act: -0.1700

Data Index: 8817851 (Open Web Text)

Max Activating Token Index: 594

Text #12

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7579. Min Act: -0.1700

Data Index: 1384102 (Open Web Text)

Max Activating Token Index: 403

Text #13

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.6993. Min Act: -0.1700

Data Index: 6112507 (Open Web Text)

Max Activating Token Index: 728

Text #14

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7311. Min Act: -0.1700

Data Index: 8155091 (Open Web Text)

Max Activating Token Index: 607

Text #15

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.6992. Min Act: -0.1700

Data Index: 8736783 (Open Web Text)

Max Activating Token Index: 548

Text #16

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7432. Min Act: -0.1700

Data Index: 8086605 (Open Web Text)

Max Activating Token Index: 611

Text #17

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7244. Min Act: -0.1700

Data Index: 3846527 (Open Web Text)

Max Activating Token Index: 841

Text #18

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7432. Min Act: -0.1700

Data Index: 7400228 (Open Web Text)

Max Activating Token Index: 256

Text #19

Max Range: 3.0382. Min Range: -3.0382

Max Act: 2.7168. Min Act: -0.1700

Data Index: 3624130 (Open Web Text)

Max Activating Token Index: 293