Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2760 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1945. Min Act: -0.1700

Data Index: 5594263 (Open Web Text)

Max Activating Token Index: 564

Text #1

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1808. Min Act: -0.1700

Data Index: 3048557 (Open Web Text)

Max Activating Token Index: 886

Text #2

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1607. Min Act: -0.1700

Data Index: 4440551 (Open Web Text)

Max Activating Token Index: 68

Text #3

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1359. Min Act: -0.1699

Data Index: 3660857 (Open Web Text)

Max Activating Token Index: 989

Text #4

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1466. Min Act: -0.1700

Data Index: 1863262 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1505. Min Act: -0.1700

Data Index: 849477 (Open Web Text)

Max Activating Token Index: 321

Text #6

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1487. Min Act: -0.1700

Data Index: 7293814 (Open Web Text)

Max Activating Token Index: 747

Text #7

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1229. Min Act: -0.1700

Data Index: 2322656 (Open Web Text)

Max Activating Token Index: 577

Text #8

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1230. Min Act: -0.1700

Data Index: 6494798 (Open Web Text)

Max Activating Token Index: 851

Text #9

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1229. Min Act: -0.1700

Data Index: 7082291 (Open Web Text)

Max Activating Token Index: 883

Text #10

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1224. Min Act: -0.1700

Data Index: 5464886 (Open Web Text)

Max Activating Token Index: 583

Text #11

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1227. Min Act: -0.1700

Data Index: 5881073 (Open Web Text)

Max Activating Token Index: 411

Text #12

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.1213. Min Act: -0.1700

Data Index: 5643724 (Open Web Text)

Max Activating Token Index: 277

Text #13

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0946. Min Act: -0.1700

Data Index: 4264463 (Open Web Text)

Max Activating Token Index: 842

Text #14

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0924. Min Act: -0.1700

Data Index: 8222653 (Open Web Text)

Max Activating Token Index: 488

Text #15

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0912. Min Act: -0.1700

Data Index: 8651293 (Open Web Text)

Max Activating Token Index: 581

Text #16

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0727. Min Act: -0.1700

Data Index: 2274512 (Open Web Text)

Max Activating Token Index: 128

Text #17

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0920. Min Act: -0.1700

Data Index: 6870725 (Open Web Text)

Max Activating Token Index: 808

Text #18

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0691. Min Act: -0.1700

Data Index: 215321 (Open Web Text)

Max Activating Token Index: 500

Text #19

Max Range: 2.1945. Min Range: -2.1945

Max Act: 2.0647. Min Act: -0.1700

Data Index: 7039938 (Open Web Text)

Max Activating Token Index: 65