Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 44 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8936. Min Act: -0.1700

Data Index: 3605018 (Open Web Text)

Max Activating Token Index: 781

Text #1

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8134. Min Act: -0.1700

Data Index: 950499 (Open Web Text)

Max Activating Token Index: 858

Text #2

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8323. Min Act: -0.1700

Data Index: 6214811 (Open Web Text)

Max Activating Token Index: 817

Text #3

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8499. Min Act: -0.1700

Data Index: 749687 (Open Web Text)

Max Activating Token Index: 206

Text #4

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8641. Min Act: -0.1700

Data Index: 3146057 (Open Web Text)

Max Activating Token Index: 958

Text #5

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8336. Min Act: -0.1700

Data Index: 3827087 (Open Web Text)

Max Activating Token Index: 833

Text #6

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8130. Min Act: -0.1700

Data Index: 4677045 (Open Web Text)

Max Activating Token Index: 830

Text #7

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7943. Min Act: -0.1700

Data Index: 1591018 (Open Web Text)

Max Activating Token Index: 483

Text #8

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8347. Min Act: -0.1700

Data Index: 694598 (Open Web Text)

Max Activating Token Index: 298

Text #9

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8269. Min Act: -0.1700

Data Index: 4856611 (Open Web Text)

Max Activating Token Index: 116

Text #10

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7318. Min Act: -0.1700

Data Index: 6949854 (Open Web Text)

Max Activating Token Index: 765

Text #11

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8715. Min Act: -0.1700

Data Index: 3088284 (Open Web Text)

Max Activating Token Index: 857

Text #12

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8015. Min Act: -0.1700

Data Index: 6680990 (Open Web Text)

Max Activating Token Index: 859

Text #13

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7715. Min Act: -0.1700

Data Index: 3462019 (Open Web Text)

Max Activating Token Index: 991

Text #14

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7478. Min Act: -0.1700

Data Index: 3202713 (Open Web Text)

Max Activating Token Index: 694

Text #15

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.8050. Min Act: -0.1700

Data Index: 1083024 (Open Web Text)

Max Activating Token Index: 325

Text #16

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7695. Min Act: -0.1700

Data Index: 226070 (Open Web Text)

Max Activating Token Index: 818

Text #17

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7919. Min Act: -0.1700

Data Index: 4749700 (Open Web Text)

Max Activating Token Index: 630

Text #18

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7814. Min Act: -0.1700

Data Index: 1644083 (Open Web Text)

Max Activating Token Index: 780

Text #19

Max Range: 3.8936. Min Range: -3.8936

Max Act: 3.7373. Min Act: -0.1700

Data Index: 1807120 (Open Web Text)

Max Activating Token Index: 833