Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 949 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2976. Min Range: -3.2976

Max Act: 3.2976. Min Act: -0.1700

Data Index: 8810024 (Open Web Text)

Max Activating Token Index: 44

Text #1

Max Range: 3.2976. Min Range: -3.2976

Max Act: 3.1637. Min Act: -0.1700

Data Index: 4855845 (Open Web Text)

Max Activating Token Index: 235

Text #2

Max Range: 3.2976. Min Range: -3.2976

Max Act: 3.0999. Min Act: -0.1700

Data Index: 294174 (Open Web Text)

Max Activating Token Index: 500

Text #3

Max Range: 3.2976. Min Range: -3.2976

Max Act: 3.0218. Min Act: -0.1700

Data Index: 5556059 (Open Web Text)

Max Activating Token Index: 138

Text #4

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9352. Min Act: -0.1700

Data Index: 5178357 (Open Web Text)

Max Activating Token Index: 107

Text #5

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9879. Min Act: -0.1700

Data Index: 1508219 (Open Web Text)

Max Activating Token Index: 19

Text #6

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9401. Min Act: -0.1700

Data Index: 2656119 (Open Web Text)

Max Activating Token Index: 838

Text #7

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9529. Min Act: -0.1700

Data Index: 944932 (Open Web Text)

Max Activating Token Index: 52

Text #8

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9427. Min Act: -0.1700

Data Index: 8390217 (Open Web Text)

Max Activating Token Index: 988

Text #9

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9273. Min Act: -0.1700

Data Index: 321437 (Open Web Text)

Max Activating Token Index: 953

Text #10

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9316. Min Act: -0.1700

Data Index: 2240311 (Open Web Text)

Max Activating Token Index: 997

Text #11

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9564. Min Act: -0.1700

Data Index: 2289333 (Open Web Text)

Max Activating Token Index: 144

Text #12

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9091. Min Act: -0.1700

Data Index: 6179885 (Open Web Text)

Max Activating Token Index: 136

Text #13

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9372. Min Act: -0.1700

Data Index: 8535775 (Open Web Text)

Max Activating Token Index: 267

Text #14

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9125. Min Act: -0.1700

Data Index: 7826191 (Open Web Text)

Max Activating Token Index: 561

Text #15

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9081. Min Act: -0.1700

Data Index: 7305698 (Open Web Text)

Max Activating Token Index: 312

Text #16

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.8935. Min Act: -0.1700

Data Index: 6335016 (Open Web Text)

Max Activating Token Index: 258

Text #17

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.9084. Min Act: -0.1700

Data Index: 8704931 (Open Web Text)

Max Activating Token Index: 408

Text #18

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.8853. Min Act: -0.1700

Data Index: 7275108 (Open Web Text)

Max Activating Token Index: 622

Text #19

Max Range: 3.2976. Min Range: -3.2976

Max Act: 2.8714. Min Act: -0.1700

Data Index: 5144290 (Open Web Text)

Max Activating Token Index: 528