Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3063 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0936. Min Range: -3.0936

Max Act: 3.0936. Min Act: -0.1700

Data Index: 1755983 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.7096. Min Act: -0.1700

Data Index: 7720466 (Open Web Text)

Max Activating Token Index: 255

Text #2

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.7147. Min Act: -0.1700

Data Index: 4138241 (Open Web Text)

Max Activating Token Index: 373

Text #3

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6993. Min Act: -0.1700

Data Index: 4020350 (Open Web Text)

Max Activating Token Index: 17

Text #4

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6536. Min Act: -0.1700

Data Index: 3965397 (Open Web Text)

Max Activating Token Index: 532

Text #5

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6711. Min Act: -0.1700

Data Index: 1134853 (Open Web Text)

Max Activating Token Index: 251

Text #6

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6675. Min Act: -0.1700

Data Index: 6674105 (Open Web Text)

Max Activating Token Index: 677

Text #7

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6249. Min Act: -0.1700

Data Index: 4109773 (Open Web Text)

Max Activating Token Index: 18

Text #8

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6633. Min Act: -0.1700

Data Index: 1855674 (Open Web Text)

Max Activating Token Index: 782

Text #9

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6064. Min Act: -0.1700

Data Index: 8255750 (Open Web Text)

Max Activating Token Index: 22

Text #10

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6195. Min Act: -0.1700

Data Index: 3829269 (Open Web Text)

Max Activating Token Index: 179

Text #11

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.6039. Min Act: -0.1700

Data Index: 853238 (Open Web Text)

Max Activating Token Index: 419

Text #12

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5783. Min Act: -0.1642

Data Index: 7080103 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5941. Min Act: -0.1700

Data Index: 3370180 (Open Web Text)

Max Activating Token Index: 169

Text #14

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5844. Min Act: -0.1700

Data Index: 895159 (Open Web Text)

Max Activating Token Index: 441

Text #15

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5783. Min Act: -0.1700

Data Index: 3758224 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5783. Min Act: -0.1700

Data Index: 257270 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5499. Min Act: -0.1700

Data Index: 1949019 (Open Web Text)

Max Activating Token Index: 417

Text #18

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5870. Min Act: -0.1700

Data Index: 7614505 (Open Web Text)

Max Activating Token Index: 649

Text #19

Max Range: 3.0936. Min Range: -3.0936

Max Act: 2.5758. Min Act: -0.1700

Data Index: 3200918 (Open Web Text)

Max Activating Token Index: 916