Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 678 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8839. Min Act: -0.1700

Data Index: 5227161 (Open Web Text)

Max Activating Token Index: 849

Text #1

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.9025. Min Act: -0.1700

Data Index: 872439 (Open Web Text)

Max Activating Token Index: 556

Text #2

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8430. Min Act: -0.1700

Data Index: 8062181 (Open Web Text)

Max Activating Token Index: 668

Text #3

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8283. Min Act: -0.1700

Data Index: 5030787 (Open Web Text)

Max Activating Token Index: 976

Text #4

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8024. Min Act: -0.1700

Data Index: 5106914 (Open Web Text)

Max Activating Token Index: 770

Text #5

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7926. Min Act: -0.1700

Data Index: 511706 (Open Web Text)

Max Activating Token Index: 79

Text #6

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7934. Min Act: -0.1700

Data Index: 8321871 (Open Web Text)

Max Activating Token Index: 780

Text #7

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8064. Min Act: -0.1700

Data Index: 2660238 (Open Web Text)

Max Activating Token Index: 965

Text #8

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7974. Min Act: -0.1700

Data Index: 8518130 (Open Web Text)

Max Activating Token Index: 940

Text #9

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.8065. Min Act: -0.1700

Data Index: 323292 (Open Web Text)

Max Activating Token Index: 744

Text #10

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7761. Min Act: -0.1700

Data Index: 6625331 (Open Web Text)

Max Activating Token Index: 411

Text #11

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7995. Min Act: -0.1700

Data Index: 3059343 (Open Web Text)

Max Activating Token Index: 96

Text #12

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7688. Min Act: -0.1700

Data Index: 4596767 (Open Web Text)

Max Activating Token Index: 390

Text #13

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7698. Min Act: -0.1700

Data Index: 4655350 (Open Web Text)

Max Activating Token Index: 231

Text #14

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7745. Min Act: -0.1700

Data Index: 446282 (Open Web Text)

Max Activating Token Index: 634

Text #15

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7848. Min Act: -0.1700

Data Index: 8700828 (Open Web Text)

Max Activating Token Index: 277

Text #16

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7785. Min Act: -0.1700

Data Index: 8666784 (Open Web Text)

Max Activating Token Index: 71

Text #17

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7602. Min Act: -0.1700

Data Index: 4081920 (Open Web Text)

Max Activating Token Index: 921

Text #18

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7513. Min Act: -0.1700

Data Index: 1962617 (Open Web Text)

Max Activating Token Index: 919

Text #19

Max Range: 2.9025. Min Range: -2.9025

Max Act: 2.7541. Min Act: -0.1700

Data Index: 4890162 (Open Web Text)

Max Activating Token Index: 82