Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1035 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5818. Min Act: -0.0990

Data Index: 627310 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5780. Min Act: -0.0183

Data Index: 8061344 (Open Web Text)

Max Activating Token Index: 93

Text #2

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5324. Min Act: 0.0233

Data Index: 584031 (Open Web Text)

Max Activating Token Index: 683

Text #3

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5188. Min Act: -0.1450

Data Index: 879620 (Open Web Text)

Max Activating Token Index: 501

Text #4

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5127. Min Act: -0.0276

Data Index: 5621055 (Open Web Text)

Max Activating Token Index: 968

Text #5

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5108. Min Act: 0.0233

Data Index: 5490082 (Open Web Text)

Max Activating Token Index: 931

Text #6

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.5138. Min Act: -0.0546

Data Index: 2849930 (Open Web Text)

Max Activating Token Index: 938

Text #7

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4936. Min Act: -0.1335

Data Index: 5713495 (Open Web Text)

Max Activating Token Index: 873

Text #8

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4918. Min Act: -0.0803

Data Index: 6500392 (Open Web Text)

Max Activating Token Index: 608

Text #9

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4980. Min Act: 0.0233

Data Index: 6052452 (Open Web Text)

Max Activating Token Index: 844

Text #10

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4957. Min Act: -0.1432

Data Index: 5373593 (Open Web Text)

Max Activating Token Index: 437

Text #11

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4886. Min Act: -0.0364

Data Index: 8656036 (Open Web Text)

Max Activating Token Index: 565

Text #12

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4873. Min Act: -0.0230

Data Index: 83069 (Open Web Text)

Max Activating Token Index: 1019

Text #13

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4888. Min Act: -0.0172

Data Index: 3650164 (Open Web Text)

Max Activating Token Index: 216

Text #14

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4740. Min Act: 0.0233

Data Index: 1802578 (Open Web Text)

Max Activating Token Index: 434

Text #15

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4804. Min Act: 0.0060

Data Index: 3256833 (Open Web Text)

Max Activating Token Index: 963

Text #16

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4860. Min Act: 0.0233

Data Index: 5619450 (Open Web Text)

Max Activating Token Index: 405

Text #17

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4748. Min Act: -0.0582

Data Index: 1532755 (Open Web Text)

Max Activating Token Index: 241

Text #18

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4763. Min Act: 0.0233

Data Index: 6052455 (Open Web Text)

Max Activating Token Index: 437

Text #19

Max Range: 1.5818. Min Range: -1.5818

Max Act: 1.4685. Min Act: 0.0124

Data Index: 6448298 (Open Web Text)

Max Activating Token Index: 1014