Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 35 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.3589. Min Act: -0.1700

Data Index: 6151187 (Open Web Text)

Max Activating Token Index: 931

Text #1

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.3549. Min Act: -0.1700

Data Index: 3747212 (Open Web Text)

Max Activating Token Index: 144

Text #2

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2787. Min Act: -0.1700

Data Index: 5147983 (Open Web Text)

Max Activating Token Index: 516

Text #3

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2603. Min Act: -0.1700

Data Index: 7546319 (Open Web Text)

Max Activating Token Index: 109

Text #4

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2130. Min Act: -0.1700

Data Index: 6393578 (Open Web Text)

Max Activating Token Index: 788

Text #5

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2416. Min Act: -0.1700

Data Index: 3171894 (Open Web Text)

Max Activating Token Index: 861

Text #6

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2230. Min Act: -0.1700

Data Index: 1025042 (Open Web Text)

Max Activating Token Index: 903

Text #7

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2263. Min Act: -0.1700

Data Index: 8360617 (Open Web Text)

Max Activating Token Index: 632

Text #8

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1878. Min Act: -0.1700

Data Index: 1033187 (Open Web Text)

Max Activating Token Index: 681

Text #9

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2102. Min Act: -0.1700

Data Index: 5147982 (Open Web Text)

Max Activating Token Index: 720

Text #10

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.2218. Min Act: -0.1700

Data Index: 6399972 (Open Web Text)

Max Activating Token Index: 831

Text #11

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1947. Min Act: -0.1700

Data Index: 4178709 (Open Web Text)

Max Activating Token Index: 468

Text #12

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1629. Min Act: -0.1700

Data Index: 772598 (Open Web Text)

Max Activating Token Index: 807

Text #13

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1549. Min Act: -0.1700

Data Index: 471708 (Open Web Text)

Max Activating Token Index: 71

Text #14

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1640. Min Act: -0.1700

Data Index: 3129488 (Open Web Text)

Max Activating Token Index: 353

Text #15

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1526. Min Act: -0.1700

Data Index: 607810 (Open Web Text)

Max Activating Token Index: 109

Text #16

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1575. Min Act: -0.1700

Data Index: 904272 (Open Web Text)

Max Activating Token Index: 315

Text #17

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1745. Min Act: -0.1700

Data Index: 1378688 (Open Web Text)

Max Activating Token Index: 447

Text #18

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1301. Min Act: -0.1700

Data Index: 907506 (Open Web Text)

Max Activating Token Index: 362

Text #19

Max Range: 3.3589. Min Range: -3.3589

Max Act: 3.1406. Min Act: -0.1700

Data Index: 7297224 (Open Web Text)

Max Activating Token Index: 350