Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1610 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0701. Min Range: -4.0701

Max Act: 4.0613. Min Act: -0.1700

Data Index: 6741989 (Open Web Text)

Max Activating Token Index: 691

Text #1

Max Range: 4.0701. Min Range: -4.0701

Max Act: 4.0701. Min Act: -0.1700

Data Index: 6847179 (Open Web Text)

Max Activating Token Index: 543

Text #2

Max Range: 4.0701. Min Range: -4.0701

Max Act: 4.0087. Min Act: -0.1700

Data Index: 1798007 (Open Web Text)

Max Activating Token Index: 761

Text #3

Max Range: 4.0701. Min Range: -4.0701

Max Act: 4.0064. Min Act: -0.1700

Data Index: 1895777 (Open Web Text)

Max Activating Token Index: 399

Text #4

Max Range: 4.0701. Min Range: -4.0701

Max Act: 4.0188. Min Act: -0.1700

Data Index: 196228 (Open Web Text)

Max Activating Token Index: 676

Text #5

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9711. Min Act: -0.1700

Data Index: 3335752 (Open Web Text)

Max Activating Token Index: 573

Text #6

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9698. Min Act: -0.1700

Data Index: 3338244 (Open Web Text)

Max Activating Token Index: 583

Text #7

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9643. Min Act: -0.1700

Data Index: 1926323 (Open Web Text)

Max Activating Token Index: 1003

Text #8

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9622. Min Act: -0.1700

Data Index: 5945207 (Open Web Text)

Max Activating Token Index: 692

Text #9

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9548. Min Act: -0.1700

Data Index: 5852374 (Open Web Text)

Max Activating Token Index: 994

Text #10

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9885. Min Act: -0.1700

Data Index: 5322225 (Open Web Text)

Max Activating Token Index: 766

Text #11

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9384. Min Act: -0.1700

Data Index: 3028301 (Open Web Text)

Max Activating Token Index: 907

Text #12

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9458. Min Act: -0.1700

Data Index: 752767 (Open Web Text)

Max Activating Token Index: 997

Text #13

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9692. Min Act: -0.1700

Data Index: 7732838 (Open Web Text)

Max Activating Token Index: 708

Text #14

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9906. Min Act: -0.1700

Data Index: 5251550 (Open Web Text)

Max Activating Token Index: 818

Text #15

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9679. Min Act: -0.1700

Data Index: 4180441 (Open Web Text)

Max Activating Token Index: 695

Text #16

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9132. Min Act: -0.1700

Data Index: 5493014 (Open Web Text)

Max Activating Token Index: 765

Text #17

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9397. Min Act: -0.1700

Data Index: 75366 (Open Web Text)

Max Activating Token Index: 694

Text #18

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9675. Min Act: -0.1700

Data Index: 2956874 (Open Web Text)

Max Activating Token Index: 549

Text #19

Max Range: 4.0701. Min Range: -4.0701

Max Act: 3.9282. Min Act: -0.1700

Data Index: 599672 (Open Web Text)

Max Activating Token Index: 722