Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 889 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.4602. Min Act: -0.1700

Data Index: 8241017 (Open Web Text)

Max Activating Token Index: 493

Text #1

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.4292. Min Act: -0.1700

Data Index: 1198368 (Open Web Text)

Max Activating Token Index: 396

Text #2

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3934. Min Act: -0.1700

Data Index: 7517727 (Open Web Text)

Max Activating Token Index: 890

Text #3

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3677. Min Act: -0.1700

Data Index: 7390317 (Open Web Text)

Max Activating Token Index: 967

Text #4

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3410. Min Act: -0.1700

Data Index: 1764961 (Open Web Text)

Max Activating Token Index: 859

Text #5

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3428. Min Act: -0.1700

Data Index: 4042807 (Open Web Text)

Max Activating Token Index: 603

Text #6

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3106. Min Act: -0.1700

Data Index: 3535846 (Open Web Text)

Max Activating Token Index: 79

Text #7

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.3089. Min Act: -0.1700

Data Index: 5965045 (Open Web Text)

Max Activating Token Index: 899

Text #8

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2628. Min Act: -0.1700

Data Index: 3661716 (Open Web Text)

Max Activating Token Index: 924

Text #9

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2945. Min Act: -0.1700

Data Index: 1094703 (Open Web Text)

Max Activating Token Index: 763

Text #10

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2684. Min Act: -0.1700

Data Index: 4836461 (Open Web Text)

Max Activating Token Index: 912

Text #11

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2705. Min Act: -0.1700

Data Index: 7498878 (Open Web Text)

Max Activating Token Index: 494

Text #12

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2697. Min Act: -0.1700

Data Index: 4846502 (Open Web Text)

Max Activating Token Index: 286

Text #13

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2545. Min Act: -0.1700

Data Index: 880712 (Open Web Text)

Max Activating Token Index: 651

Text #14

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2283. Min Act: -0.1700

Data Index: 6417163 (Open Web Text)

Max Activating Token Index: 499

Text #15

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2435. Min Act: -0.1700

Data Index: 4406812 (Open Web Text)

Max Activating Token Index: 693

Text #16

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2590. Min Act: -0.1700

Data Index: 835001 (Open Web Text)

Max Activating Token Index: 933

Text #17

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2521. Min Act: -0.1700

Data Index: 1014642 (Open Web Text)

Max Activating Token Index: 300

Text #18

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2260. Min Act: -0.1700

Data Index: 1475470 (Open Web Text)

Max Activating Token Index: 854

Text #19

Max Range: 3.4602. Min Range: -3.4602

Max Act: 3.2016. Min Act: -0.1700

Data Index: 5996768 (Open Web Text)

Max Activating Token Index: 326