Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 835 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.7851. Min Act: -0.1700

Data Index: 8378108 (Open Web Text)

Max Activating Token Index: 871

Text #1

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.6823. Min Act: -0.1700

Data Index: 1827114 (Open Web Text)

Max Activating Token Index: 340

Text #2

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.7689. Min Act: -0.1700

Data Index: 6737492 (Open Web Text)

Max Activating Token Index: 503

Text #3

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.6204. Min Act: -0.1700

Data Index: 347245 (Open Web Text)

Max Activating Token Index: 842

Text #4

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.6955. Min Act: -0.1700

Data Index: 7472046 (Open Web Text)

Max Activating Token Index: 810

Text #5

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.6402. Min Act: -0.1700

Data Index: 2161014 (Open Web Text)

Max Activating Token Index: 971

Text #6

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.6053. Min Act: -0.1700

Data Index: 2066730 (Open Web Text)

Max Activating Token Index: 52

Text #7

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.5312. Min Act: -0.1700

Data Index: 103109 (Open Web Text)

Max Activating Token Index: 133

Text #8

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.5121. Min Act: -0.1700

Data Index: 5672313 (Open Web Text)

Max Activating Token Index: 772

Text #9

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.4092. Min Act: -0.1700

Data Index: 3149045 (Open Web Text)

Max Activating Token Index: 1019

Text #10

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.4376. Min Act: -0.1700

Data Index: 4845930 (Open Web Text)

Max Activating Token Index: 334

Text #11

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3909. Min Act: -0.1700

Data Index: 6635679 (Open Web Text)

Max Activating Token Index: 471

Text #12

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3107. Min Act: -0.1700

Data Index: 6057187 (Open Web Text)

Max Activating Token Index: 385

Text #13

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3421. Min Act: -0.1700

Data Index: 7792691 (Open Web Text)

Max Activating Token Index: 325

Text #14

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3381. Min Act: -0.1700

Data Index: 4295189 (Open Web Text)

Max Activating Token Index: 534

Text #15

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3569. Min Act: -0.1700

Data Index: 8753749 (Open Web Text)

Max Activating Token Index: 48

Text #16

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3153. Min Act: -0.1700

Data Index: 3019141 (Open Web Text)

Max Activating Token Index: 703

Text #17

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3055. Min Act: -0.1700

Data Index: 8496938 (Open Web Text)

Max Activating Token Index: 38

Text #18

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.3354. Min Act: -0.1700

Data Index: 5617476 (Open Web Text)

Max Activating Token Index: 518

Text #19

Max Range: 4.7851. Min Range: -4.7851

Max Act: 4.2688. Min Act: -0.1700

Data Index: 2808317 (Open Web Text)

Max Activating Token Index: 656