Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1675 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.3403. Min Act: -0.1700

Data Index: 1580942 (Open Web Text)

Max Activating Token Index: 293

Text #1

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.2720. Min Act: -0.1700

Data Index: 2348210 (Open Web Text)

Max Activating Token Index: 786

Text #2

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.2426. Min Act: -0.1700

Data Index: 3640638 (Open Web Text)

Max Activating Token Index: 594

Text #3

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.2525. Min Act: -0.1700

Data Index: 2771718 (Open Web Text)

Max Activating Token Index: 772

Text #4

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1997. Min Act: -0.1700

Data Index: 1798913 (Open Web Text)

Max Activating Token Index: 371

Text #5

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.2062. Min Act: -0.1700

Data Index: 3070903 (Open Web Text)

Max Activating Token Index: 913

Text #6

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.2191. Min Act: -0.1700

Data Index: 2617446 (Open Web Text)

Max Activating Token Index: 957

Text #7

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1769. Min Act: -0.1700

Data Index: 5721339 (Open Web Text)

Max Activating Token Index: 500

Text #8

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1800. Min Act: -0.1700

Data Index: 8304372 (Open Web Text)

Max Activating Token Index: 527

Text #9

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1567. Min Act: -0.1700

Data Index: 4548609 (Open Web Text)

Max Activating Token Index: 931

Text #10

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1961. Min Act: -0.1700

Data Index: 8312282 (Open Web Text)

Max Activating Token Index: 915

Text #11

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1607. Min Act: -0.1700

Data Index: 5853406 (Open Web Text)

Max Activating Token Index: 483

Text #12

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1303. Min Act: -0.1700

Data Index: 4601142 (Open Web Text)

Max Activating Token Index: 869

Text #13

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1467. Min Act: -0.1700

Data Index: 5283926 (Open Web Text)

Max Activating Token Index: 691

Text #14

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1443. Min Act: -0.1700

Data Index: 2549400 (Open Web Text)

Max Activating Token Index: 847

Text #15

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1518. Min Act: -0.1700

Data Index: 5132415 (Open Web Text)

Max Activating Token Index: 458

Text #16

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1262. Min Act: -0.1700

Data Index: 1963700 (Open Web Text)

Max Activating Token Index: 868

Text #17

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1126. Min Act: -0.1700

Data Index: 6086276 (Open Web Text)

Max Activating Token Index: 539

Text #18

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.1076. Min Act: -0.1700

Data Index: 4675988 (Open Web Text)

Max Activating Token Index: 268

Text #19

Max Range: 3.3403. Min Range: -3.3403

Max Act: 3.0975. Min Act: -0.1700

Data Index: 2694103 (Open Web Text)

Max Activating Token Index: 742