Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 565 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1989. Min Range: -3.1989

Max Act: 3.1989. Min Act: -0.1700

Data Index: 6269051 (Open Web Text)

Max Activating Token Index: 724

Text #1

Max Range: 3.1989. Min Range: -3.1989

Max Act: 3.1172. Min Act: -0.1700

Data Index: 7447806 (Open Web Text)

Max Activating Token Index: 832

Text #2

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9857. Min Act: -0.1700

Data Index: 2745050 (Open Web Text)

Max Activating Token Index: 900

Text #3

Max Range: 3.1989. Min Range: -3.1989

Max Act: 3.0053. Min Act: -0.1700

Data Index: 645268 (Open Web Text)

Max Activating Token Index: 922

Text #4

Max Range: 3.1989. Min Range: -3.1989

Max Act: 3.0161. Min Act: -0.1700

Data Index: 4320056 (Open Web Text)

Max Activating Token Index: 204

Text #5

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9909. Min Act: -0.1700

Data Index: 7788797 (Open Web Text)

Max Activating Token Index: 574

Text #6

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9986. Min Act: -0.1700

Data Index: 8640557 (Open Web Text)

Max Activating Token Index: 159

Text #7

Max Range: 3.1989. Min Range: -3.1989

Max Act: 3.0026. Min Act: -0.1700

Data Index: 6870206 (Open Web Text)

Max Activating Token Index: 940

Text #8

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9993. Min Act: -0.1700

Data Index: 1203059 (Open Web Text)

Max Activating Token Index: 743

Text #9

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9823. Min Act: -0.1700

Data Index: 8683882 (Open Web Text)

Max Activating Token Index: 884

Text #10

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9937. Min Act: -0.1700

Data Index: 5095465 (Open Web Text)

Max Activating Token Index: 902

Text #11

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9671. Min Act: -0.1700

Data Index: 1224078 (Open Web Text)

Max Activating Token Index: 571

Text #12

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9761. Min Act: -0.1700

Data Index: 7461304 (Open Web Text)

Max Activating Token Index: 425

Text #13

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9810. Min Act: -0.1700

Data Index: 567973 (Open Web Text)

Max Activating Token Index: 359

Text #14

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9857. Min Act: -0.1700

Data Index: 3447336 (Open Web Text)

Max Activating Token Index: 327

Text #15

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9651. Min Act: -0.1700

Data Index: 1217424 (Open Web Text)

Max Activating Token Index: 407

Text #16

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9607. Min Act: -0.1700

Data Index: 7348231 (Open Web Text)

Max Activating Token Index: 105

Text #17

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9347. Min Act: -0.1700

Data Index: 590289 (Open Web Text)

Max Activating Token Index: 767

Text #18

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9605. Min Act: -0.1700

Data Index: 4631508 (Open Web Text)

Max Activating Token Index: 428

Text #19

Max Range: 3.1989. Min Range: -3.1989

Max Act: 2.9486. Min Act: -0.1700

Data Index: 3294135 (Open Web Text)

Max Activating Token Index: 120