Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2894 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.4904. Min Act: -0.1700

Data Index: 4659187 (Open Web Text)

Max Activating Token Index: 903

Text #1

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.3234. Min Act: -0.1700

Data Index: 8477086 (Open Web Text)

Max Activating Token Index: 670

Text #2

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.3026. Min Act: -0.1700

Data Index: 1790788 (Open Web Text)

Max Activating Token Index: 375

Text #3

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.2865. Min Act: -0.1700

Data Index: 7142863 (Open Web Text)

Max Activating Token Index: 367

Text #4

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.2590. Min Act: -0.1700

Data Index: 715843 (Open Web Text)

Max Activating Token Index: 900

Text #5

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.2006. Min Act: -0.1700

Data Index: 6578871 (Open Web Text)

Max Activating Token Index: 847

Text #6

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.2061. Min Act: -0.1700

Data Index: 6748339 (Open Web Text)

Max Activating Token Index: 990

Text #7

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.1704. Min Act: -0.1700

Data Index: 5258010 (Open Web Text)

Max Activating Token Index: 547

Text #8

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.1272. Min Act: -0.1700

Data Index: 4032499 (Open Web Text)

Max Activating Token Index: 848

Text #9

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.1598. Min Act: -0.1700

Data Index: 3831622 (Open Web Text)

Max Activating Token Index: 895

Text #10

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.1237. Min Act: -0.1700

Data Index: 1017218 (Open Web Text)

Max Activating Token Index: 917

Text #11

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0340. Min Act: -0.1700

Data Index: 3464188 (Open Web Text)

Max Activating Token Index: 542

Text #12

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0090. Min Act: -0.1700

Data Index: 2800534 (Open Web Text)

Max Activating Token Index: 908

Text #13

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0439. Min Act: -0.1700

Data Index: 6250425 (Open Web Text)

Max Activating Token Index: 394

Text #14

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0356. Min Act: -0.1700

Data Index: 3371572 (Open Web Text)

Max Activating Token Index: 326

Text #15

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0171. Min Act: -0.1700

Data Index: 5605410 (Open Web Text)

Max Activating Token Index: 975

Text #16

Max Range: 4.4904. Min Range: -4.4904

Max Act: 3.9906. Min Act: -0.1700

Data Index: 3788820 (Open Web Text)

Max Activating Token Index: 402

Text #17

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0520. Min Act: -0.1700

Data Index: 3441555 (Open Web Text)

Max Activating Token Index: 788

Text #18

Max Range: 4.4904. Min Range: -4.4904

Max Act: 4.0350. Min Act: -0.1700

Data Index: 315080 (Open Web Text)

Max Activating Token Index: 691

Text #19

Max Range: 4.4904. Min Range: -4.4904

Max Act: 3.9910. Min Act: -0.1700

Data Index: 5499340 (Open Web Text)

Max Activating Token Index: 755