Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2923 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2863. Min Act: -0.1700

Data Index: 4307502 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2818. Min Act: -0.1700

Data Index: 390782 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2594. Min Act: -0.1700

Data Index: 1614512 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2677. Min Act: -0.1700

Data Index: 6407690 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2389. Min Act: -0.1700

Data Index: 2366661 (Open Web Text)

Max Activating Token Index: 384

Text #5

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2364. Min Act: -0.1700

Data Index: 874834 (Open Web Text)

Max Activating Token Index: 8

Text #6

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2333. Min Act: -0.1700

Data Index: 3969582 (Open Web Text)

Max Activating Token Index: 470

Text #7

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2105. Min Act: -0.1700

Data Index: 3969584 (Open Web Text)

Max Activating Token Index: 848

Text #8

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2101. Min Act: -0.1700

Data Index: 7340764 (Open Web Text)

Max Activating Token Index: 16

Text #9

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2140. Min Act: -0.1700

Data Index: 1225607 (Open Web Text)

Max Activating Token Index: 23

Text #10

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2009. Min Act: -0.1700

Data Index: 5643860 (Open Web Text)

Max Activating Token Index: 969

Text #11

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.2062. Min Act: -0.1700

Data Index: 6676838 (Open Web Text)

Max Activating Token Index: 958

Text #12

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1680. Min Act: -0.1700

Data Index: 164990 (Open Web Text)

Max Activating Token Index: 961

Text #13

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1748. Min Act: -0.1700

Data Index: 3964556 (Open Web Text)

Max Activating Token Index: 968

Text #14

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1856. Min Act: -0.1700

Data Index: 5651327 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1713. Min Act: -0.1700

Data Index: 4533736 (Open Web Text)

Max Activating Token Index: 534

Text #16

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1653. Min Act: -0.1700

Data Index: 3485445 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1645. Min Act: -0.1700

Data Index: 3969578 (Open Web Text)

Max Activating Token Index: 822

Text #18

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1600. Min Act: -0.1700

Data Index: 1338140 (Open Web Text)

Max Activating Token Index: 32

Text #19

Max Range: 3.2863. Min Range: -3.2863

Max Act: 3.1593. Min Act: -0.1700

Data Index: 6473830 (Open Web Text)

Max Activating Token Index: 63