Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1945 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.5709. Min Act: -0.1700

Data Index: 3078760 (Open Web Text)

Max Activating Token Index: 928

Text #1

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.5143. Min Act: -0.1700

Data Index: 1721776 (Open Web Text)

Max Activating Token Index: 850

Text #2

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.5176. Min Act: -0.1700

Data Index: 4246528 (Open Web Text)

Max Activating Token Index: 860

Text #3

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.3870. Min Act: -0.1700

Data Index: 1721779 (Open Web Text)

Max Activating Token Index: 972

Text #4

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.3708. Min Act: -0.1700

Data Index: 7295936 (Open Web Text)

Max Activating Token Index: 911

Text #5

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.2877. Min Act: -0.1700

Data Index: 4871520 (Open Web Text)

Max Activating Token Index: 886

Text #6

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.2596. Min Act: -0.1700

Data Index: 6792080 (Open Web Text)

Max Activating Token Index: 839

Text #7

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.2599. Min Act: -0.1700

Data Index: 8442372 (Open Web Text)

Max Activating Token Index: 805

Text #8

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1945. Min Act: -0.1700

Data Index: 4246529 (Open Web Text)

Max Activating Token Index: 869

Text #9

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.2294. Min Act: -0.1700

Data Index: 4246530 (Open Web Text)

Max Activating Token Index: 848

Text #10

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1759. Min Act: -0.1700

Data Index: 8753471 (Open Web Text)

Max Activating Token Index: 574

Text #11

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1934. Min Act: -0.1700

Data Index: 4490688 (Open Web Text)

Max Activating Token Index: 834

Text #12

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1925. Min Act: -0.1700

Data Index: 7632862 (Open Web Text)

Max Activating Token Index: 841

Text #13

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1749. Min Act: -0.1700

Data Index: 6265056 (Open Web Text)

Max Activating Token Index: 901

Text #14

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1085. Min Act: -0.1700

Data Index: 3287800 (Open Web Text)

Max Activating Token Index: 894

Text #15

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1520. Min Act: -0.1700

Data Index: 5747915 (Open Web Text)

Max Activating Token Index: 817

Text #16

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1114. Min Act: -0.1700

Data Index: 24945 (Open Web Text)

Max Activating Token Index: 921

Text #17

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.1040. Min Act: -0.1700

Data Index: 2844472 (Open Web Text)

Max Activating Token Index: 965

Text #18

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.0684. Min Act: -0.1700

Data Index: 4462906 (Open Web Text)

Max Activating Token Index: 879

Text #19

Max Range: 6.5709. Min Range: -6.5709

Max Act: 6.0834. Min Act: -0.1700

Data Index: 928672 (Open Web Text)

Max Activating Token Index: 819