Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1431 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2623. Min Act: -0.1700

Data Index: 1010868 (Open Web Text)

Max Activating Token Index: 702

Text #1

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2697. Min Act: -0.1700

Data Index: 27786 (Open Web Text)

Max Activating Token Index: 456

Text #2

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2508. Min Act: -0.1700

Data Index: 3753229 (Open Web Text)

Max Activating Token Index: 391

Text #3

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2357. Min Act: -0.1700

Data Index: 1501360 (Open Web Text)

Max Activating Token Index: 948

Text #4

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2312. Min Act: -0.1700

Data Index: 38694 (Open Web Text)

Max Activating Token Index: 821

Text #5

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1988. Min Act: -0.1700

Data Index: 7498924 (Open Web Text)

Max Activating Token Index: 833

Text #6

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.2090. Min Act: -0.1700

Data Index: 1559813 (Open Web Text)

Max Activating Token Index: 369

Text #7

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1999. Min Act: -0.1700

Data Index: 7978657 (Open Web Text)

Max Activating Token Index: 437

Text #8

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1731. Min Act: -0.1700

Data Index: 887433 (Open Web Text)

Max Activating Token Index: 789

Text #9

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1735. Min Act: -0.1700

Data Index: 2080625 (Open Web Text)

Max Activating Token Index: 310

Text #10

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1464. Min Act: -0.1700

Data Index: 6628691 (Open Web Text)

Max Activating Token Index: 25

Text #11

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1209. Min Act: -0.1700

Data Index: 2933084 (Open Web Text)

Max Activating Token Index: 905

Text #12

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1543. Min Act: -0.1700

Data Index: 1379731 (Open Web Text)

Max Activating Token Index: 592

Text #13

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1510. Min Act: -0.1700

Data Index: 1692872 (Open Web Text)

Max Activating Token Index: 577

Text #14

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1387. Min Act: -0.1700

Data Index: 971317 (Open Web Text)

Max Activating Token Index: 832

Text #15

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1366. Min Act: -0.1700

Data Index: 5949753 (Open Web Text)

Max Activating Token Index: 732

Text #16

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1162. Min Act: -0.1700

Data Index: 5778538 (Open Web Text)

Max Activating Token Index: 483

Text #17

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1213. Min Act: -0.1700

Data Index: 2040854 (Open Web Text)

Max Activating Token Index: 802

Text #18

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.1068. Min Act: -0.1700

Data Index: 3065782 (Open Web Text)

Max Activating Token Index: 396

Text #19

Max Range: 4.2697. Min Range: -4.2697

Max Act: 4.0979. Min Act: -0.1700

Data Index: 5310881 (Open Web Text)

Max Activating Token Index: 401