Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1964 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.8657. Min Act: -0.1700

Data Index: 6179741 (Open Web Text)

Max Activating Token Index: 268

Text #1

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.6642. Min Act: -0.1700

Data Index: 288397 (Open Web Text)

Max Activating Token Index: 914

Text #2

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.6305. Min Act: -0.1700

Data Index: 7186392 (Open Web Text)

Max Activating Token Index: 262

Text #3

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.6059. Min Act: -0.1700

Data Index: 267624 (Open Web Text)

Max Activating Token Index: 138

Text #4

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.6116. Min Act: -0.1700

Data Index: 6302227 (Open Web Text)

Max Activating Token Index: 499

Text #5

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5690. Min Act: -0.1700

Data Index: 4935196 (Open Web Text)

Max Activating Token Index: 327

Text #6

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5519. Min Act: -0.1700

Data Index: 8074424 (Open Web Text)

Max Activating Token Index: 117

Text #7

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5394. Min Act: -0.1700

Data Index: 7435809 (Open Web Text)

Max Activating Token Index: 800

Text #8

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5236. Min Act: -0.1700

Data Index: 5726977 (Open Web Text)

Max Activating Token Index: 541

Text #9

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5109. Min Act: -0.1700

Data Index: 8814 (Open Web Text)

Max Activating Token Index: 612

Text #10

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4882. Min Act: -0.1700

Data Index: 6183595 (Open Web Text)

Max Activating Token Index: 91

Text #11

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5079. Min Act: -0.1700

Data Index: 3668949 (Open Web Text)

Max Activating Token Index: 387

Text #12

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4841. Min Act: -0.1700

Data Index: 1388748 (Open Web Text)

Max Activating Token Index: 469

Text #13

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.5007. Min Act: -0.1700

Data Index: 5981081 (Open Web Text)

Max Activating Token Index: 83

Text #14

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4823. Min Act: -0.1700

Data Index: 7672890 (Open Web Text)

Max Activating Token Index: 729

Text #15

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4711. Min Act: -0.1700

Data Index: 5158339 (Open Web Text)

Max Activating Token Index: 711

Text #16

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4499. Min Act: -0.1700

Data Index: 8312294 (Open Web Text)

Max Activating Token Index: 445

Text #17

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4455. Min Act: -0.1700

Data Index: 8645460 (Open Web Text)

Max Activating Token Index: 947

Text #18

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4804. Min Act: -0.1700

Data Index: 8705915 (Open Web Text)

Max Activating Token Index: 841

Text #19

Max Range: 3.8657. Min Range: -3.8657

Max Act: 3.4456. Min Act: -0.1700

Data Index: 2633812 (Open Web Text)

Max Activating Token Index: 285