Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1447 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.1849. Min Act: -0.1700

Data Index: 2885960 (Open Web Text)

Max Activating Token Index: 612

Text #1

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.1561. Min Act: -0.1700

Data Index: 6273841 (Open Web Text)

Max Activating Token Index: 251

Text #2

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.1547. Min Act: -0.1700

Data Index: 5889073 (Open Web Text)

Max Activating Token Index: 50

Text #3

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.1216. Min Act: -0.1700

Data Index: 3649298 (Open Web Text)

Max Activating Token Index: 618

Text #4

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0971. Min Act: -0.1700

Data Index: 6932736 (Open Web Text)

Max Activating Token Index: 817

Text #5

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0899. Min Act: -0.1700

Data Index: 2678543 (Open Web Text)

Max Activating Token Index: 772

Text #6

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0794. Min Act: -0.1700

Data Index: 5779931 (Open Web Text)

Max Activating Token Index: 942

Text #7

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0885. Min Act: -0.1700

Data Index: 7602456 (Open Web Text)

Max Activating Token Index: 701

Text #8

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0647. Min Act: -0.1700

Data Index: 8568457 (Open Web Text)

Max Activating Token Index: 97

Text #9

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0453. Min Act: -0.1700

Data Index: 7817137 (Open Web Text)

Max Activating Token Index: 53

Text #10

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0343. Min Act: -0.1700

Data Index: 5556117 (Open Web Text)

Max Activating Token Index: 142

Text #11

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0436. Min Act: -0.1700

Data Index: 1516824 (Open Web Text)

Max Activating Token Index: 617

Text #12

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0283. Min Act: -0.1700

Data Index: 7481095 (Open Web Text)

Max Activating Token Index: 633

Text #13

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0246. Min Act: -0.1700

Data Index: 4589924 (Open Web Text)

Max Activating Token Index: 358

Text #14

Max Range: 3.1849. Min Range: -3.1849

Max Act: 2.9851. Min Act: -0.1700

Data Index: 8061339 (Open Web Text)

Max Activating Token Index: 55

Text #15

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0011. Min Act: -0.1700

Data Index: 1047225 (Open Web Text)

Max Activating Token Index: 275

Text #16

Max Range: 3.1849. Min Range: -3.1849

Max Act: 2.9921. Min Act: -0.1700

Data Index: 1595560 (Open Web Text)

Max Activating Token Index: 282

Text #17

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0098. Min Act: -0.1700

Data Index: 7546429 (Open Web Text)

Max Activating Token Index: 964

Text #18

Max Range: 3.1849. Min Range: -3.1849

Max Act: 3.0087. Min Act: -0.1700

Data Index: 4517081 (Open Web Text)

Max Activating Token Index: 787

Text #19

Max Range: 3.1849. Min Range: -3.1849

Max Act: 2.9890. Min Act: -0.1700

Data Index: 4559935 (Open Web Text)

Max Activating Token Index: 80