Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 872 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.1977. Min Act: -0.1700

Data Index: 1718389 (Open Web Text)

Max Activating Token Index: 531

Text #1

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0533. Min Act: -0.1700

Data Index: 3830449 (Open Web Text)

Max Activating Token Index: 62

Text #2

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9705. Min Act: -0.1700

Data Index: 2126222 (Open Web Text)

Max Activating Token Index: 619

Text #3

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0746. Min Act: -0.1700

Data Index: 7039613 (Open Web Text)

Max Activating Token Index: 724

Text #4

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0406. Min Act: -0.1700

Data Index: 8728877 (Open Web Text)

Max Activating Token Index: 17

Text #5

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0124. Min Act: -0.1700

Data Index: 5849755 (Open Web Text)

Max Activating Token Index: 514

Text #6

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9496. Min Act: -0.1700

Data Index: 6440798 (Open Web Text)

Max Activating Token Index: 1001

Text #7

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0295. Min Act: -0.1700

Data Index: 963037 (Open Web Text)

Max Activating Token Index: 66

Text #8

Max Range: 4.1977. Min Range: -4.1977

Max Act: 4.0199. Min Act: -0.1700

Data Index: 3178718 (Open Web Text)

Max Activating Token Index: 423

Text #9

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9457. Min Act: -0.1700

Data Index: 630816 (Open Web Text)

Max Activating Token Index: 442

Text #10

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9921. Min Act: -0.1700

Data Index: 3943412 (Open Web Text)

Max Activating Token Index: 642

Text #11

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9461. Min Act: -0.1700

Data Index: 1068244 (Open Web Text)

Max Activating Token Index: 573

Text #12

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9813. Min Act: -0.1700

Data Index: 2684419 (Open Web Text)

Max Activating Token Index: 919

Text #13

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9537. Min Act: -0.1700

Data Index: 3232727 (Open Web Text)

Max Activating Token Index: 589

Text #14

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9884. Min Act: -0.1700

Data Index: 1351784 (Open Web Text)

Max Activating Token Index: 864

Text #15

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9591. Min Act: -0.1700

Data Index: 6214409 (Open Web Text)

Max Activating Token Index: 305

Text #16

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9712. Min Act: -0.1700

Data Index: 3141561 (Open Web Text)

Max Activating Token Index: 391

Text #17

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9550. Min Act: -0.1700

Data Index: 4540130 (Open Web Text)

Max Activating Token Index: 672

Text #18

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.8739. Min Act: -0.1700

Data Index: 7279622 (Open Web Text)

Max Activating Token Index: 547

Text #19

Max Range: 4.1977. Min Range: -4.1977

Max Act: 3.9468. Min Act: -0.1700

Data Index: 8446442 (Open Web Text)

Max Activating Token Index: 285