Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 29 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0693. Min Range: -2.0693

Max Act: 2.0490. Min Act: -0.1700

Data Index: 4329483 (Open Web Text)

Max Activating Token Index: 803

Text #1

Max Range: 2.0693. Min Range: -2.0693

Max Act: 2.0693. Min Act: -0.1700

Data Index: 6522706 (Open Web Text)

Max Activating Token Index: 956

Text #2

Max Range: 2.0693. Min Range: -2.0693

Max Act: 2.0123. Min Act: -0.1700

Data Index: 3627612 (Open Web Text)

Max Activating Token Index: 573

Text #3

Max Range: 2.0693. Min Range: -2.0693

Max Act: 2.0101. Min Act: -0.1700

Data Index: 3323191 (Open Web Text)

Max Activating Token Index: 528

Text #4

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9804. Min Act: -0.1700

Data Index: 857756 (Open Web Text)

Max Activating Token Index: 52

Text #5

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9771. Min Act: -0.1700

Data Index: 5120976 (Open Web Text)

Max Activating Token Index: 494

Text #6

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9861. Min Act: -0.1700

Data Index: 1197417 (Open Web Text)

Max Activating Token Index: 15

Text #7

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9600. Min Act: -0.1700

Data Index: 5956891 (Open Web Text)

Max Activating Token Index: 14

Text #8

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9538. Min Act: -0.1700

Data Index: 1688512 (Open Web Text)

Max Activating Token Index: 145

Text #9

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9569. Min Act: -0.1700

Data Index: 2413745 (Open Web Text)

Max Activating Token Index: 338

Text #10

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9462. Min Act: -0.1700

Data Index: 5637922 (Open Web Text)

Max Activating Token Index: 731

Text #11

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9445. Min Act: -0.1700

Data Index: 7800422 (Open Web Text)

Max Activating Token Index: 220

Text #12

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9339. Min Act: -0.1700

Data Index: 5200620 (Open Web Text)

Max Activating Token Index: 46

Text #13

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9052. Min Act: -0.1700

Data Index: 6090469 (Open Web Text)

Max Activating Token Index: 878

Text #14

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9618. Min Act: -0.1700

Data Index: 7009869 (Open Web Text)

Max Activating Token Index: 634

Text #15

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9321. Min Act: -0.1700

Data Index: 8267445 (Open Web Text)

Max Activating Token Index: 422

Text #16

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9159. Min Act: -0.1700

Data Index: 4206497 (Open Web Text)

Max Activating Token Index: 160

Text #17

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9362. Min Act: -0.1700

Data Index: 195659 (Open Web Text)

Max Activating Token Index: 247

Text #18

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.8973. Min Act: -0.1700

Data Index: 1214414 (Open Web Text)

Max Activating Token Index: 95

Text #19

Max Range: 2.0693. Min Range: -2.0693

Max Act: 1.9127. Min Act: -0.1700

Data Index: 4035883 (Open Web Text)

Max Activating Token Index: 303