Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1814 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3422. Min Range: -5.3422

Max Act: 5.3422. Min Act: -0.1700

Data Index: 4712864 (Open Web Text)

Max Activating Token Index: 831

Text #1

Max Range: 5.3422. Min Range: -5.3422

Max Act: 5.1154. Min Act: -0.1700

Data Index: 2288015 (Open Web Text)

Max Activating Token Index: 718

Text #2

Max Range: 5.3422. Min Range: -5.3422

Max Act: 5.0892. Min Act: -0.1700

Data Index: 8110917 (Open Web Text)

Max Activating Token Index: 234

Text #3

Max Range: 5.3422. Min Range: -5.3422

Max Act: 5.0211. Min Act: -0.1700

Data Index: 7423966 (Open Web Text)

Max Activating Token Index: 514

Text #4

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.9922. Min Act: -0.1700

Data Index: 2411915 (Open Web Text)

Max Activating Token Index: 419

Text #5

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.9466. Min Act: -0.1700

Data Index: 8680503 (Open Web Text)

Max Activating Token Index: 181

Text #6

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.9284. Min Act: -0.1700

Data Index: 3477489 (Open Web Text)

Max Activating Token Index: 726

Text #7

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.9524. Min Act: -0.1700

Data Index: 1528019 (Open Web Text)

Max Activating Token Index: 124

Text #8

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8571. Min Act: -0.1700

Data Index: 6322159 (Open Web Text)

Max Activating Token Index: 532

Text #9

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8812. Min Act: -0.1700

Data Index: 7244036 (Open Web Text)

Max Activating Token Index: 671

Text #10

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8668. Min Act: -0.1700

Data Index: 4474536 (Open Web Text)

Max Activating Token Index: 807

Text #11

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8772. Min Act: -0.1700

Data Index: 1199499 (Open Web Text)

Max Activating Token Index: 410

Text #12

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8423. Min Act: -0.1700

Data Index: 2974865 (Open Web Text)

Max Activating Token Index: 665

Text #13

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8388. Min Act: -0.1700

Data Index: 8695049 (Open Web Text)

Max Activating Token Index: 690

Text #14

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8087. Min Act: -0.1700

Data Index: 7705904 (Open Web Text)

Max Activating Token Index: 574

Text #15

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.7947. Min Act: -0.1700

Data Index: 7223155 (Open Web Text)

Max Activating Token Index: 695

Text #16

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8225. Min Act: -0.1700

Data Index: 1931319 (Open Web Text)

Max Activating Token Index: 286

Text #17

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.8028. Min Act: -0.1700

Data Index: 2917882 (Open Web Text)

Max Activating Token Index: 569

Text #18

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.7900. Min Act: -0.1700

Data Index: 1185153 (Open Web Text)

Max Activating Token Index: 814

Text #19

Max Range: 5.3422. Min Range: -5.3422

Max Act: 4.7653. Min Act: -0.1700

Data Index: 5262615 (Open Web Text)

Max Activating Token Index: 591