Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1914 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4574. Min Act: -0.0760

Data Index: 7165969 (Open Web Text)

Max Activating Token Index: 982

Text #1

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4400. Min Act: -0.1044

Data Index: 5299626 (Open Web Text)

Max Activating Token Index: 573

Text #2

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4448. Min Act: -0.0019

Data Index: 2669306 (Open Web Text)

Max Activating Token Index: 618

Text #3

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4324. Min Act: -0.0506

Data Index: 6799900 (Open Web Text)

Max Activating Token Index: 729

Text #4

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4278. Min Act: -0.0436

Data Index: 7270618 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4330. Min Act: -0.1203

Data Index: 7314704 (Open Web Text)

Max Activating Token Index: 708

Text #6

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4202. Min Act: -0.1269

Data Index: 1582655 (Open Web Text)

Max Activating Token Index: 391

Text #7

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4162. Min Act: -0.1112

Data Index: 501792 (Open Web Text)

Max Activating Token Index: 362

Text #8

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4240. Min Act: -0.1137

Data Index: 2658147 (Open Web Text)

Max Activating Token Index: 427

Text #9

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4251. Min Act: -0.0895

Data Index: 44297 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4171. Min Act: -0.1166

Data Index: 1175621 (Open Web Text)

Max Activating Token Index: 409

Text #11

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4140. Min Act: -0.0579

Data Index: 6719239 (Open Web Text)

Max Activating Token Index: 490

Text #12

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4108. Min Act: -0.1189

Data Index: 501788 (Open Web Text)

Max Activating Token Index: 737

Text #13

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4124. Min Act: -0.0414

Data Index: 6971225 (Open Web Text)

Max Activating Token Index: 373

Text #14

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4021. Min Act: -0.0720

Data Index: 6414361 (Open Web Text)

Max Activating Token Index: 858

Text #15

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4035. Min Act: -0.0719

Data Index: 5794765 (Open Web Text)

Max Activating Token Index: 695

Text #16

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4049. Min Act: -0.0851

Data Index: 3364011 (Open Web Text)

Max Activating Token Index: 892

Text #17

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4075. Min Act: -0.0446

Data Index: 3645365 (Open Web Text)

Max Activating Token Index: 390

Text #18

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.3971. Min Act: -0.0922

Data Index: 5989281 (Open Web Text)

Max Activating Token Index: 344

Text #19

Max Range: 1.4574. Min Range: -1.4574

Max Act: 1.4038. Min Act: -0.0538

Data Index: 5279398 (Open Web Text)

Max Activating Token Index: 434