Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2723 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.7066. Min Act: -0.1700

Data Index: 2513302 (Open Web Text)

Max Activating Token Index: 128

Text #1

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.7013. Min Act: -0.1700

Data Index: 509264 (Open Web Text)

Max Activating Token Index: 179

Text #2

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6938. Min Act: -0.1700

Data Index: 1130991 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6834. Min Act: -0.1700

Data Index: 5994396 (Open Web Text)

Max Activating Token Index: 695

Text #4

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6716. Min Act: -0.1700

Data Index: 519953 (Open Web Text)

Max Activating Token Index: 162

Text #5

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6721. Min Act: -0.1700

Data Index: 8007798 (Open Web Text)

Max Activating Token Index: 150

Text #6

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6661. Min Act: -0.1699

Data Index: 2531066 (Open Web Text)

Max Activating Token Index: 241

Text #7

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6581. Min Act: -0.1673

Data Index: 5129158 (Open Web Text)

Max Activating Token Index: 682

Text #8

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6495. Min Act: -0.1700

Data Index: 7845517 (Open Web Text)

Max Activating Token Index: 123

Text #9

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6626. Min Act: -0.1699

Data Index: 1599156 (Open Web Text)

Max Activating Token Index: 824

Text #10

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6607. Min Act: -0.1700

Data Index: 832113 (Open Web Text)

Max Activating Token Index: 532

Text #11

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6502. Min Act: -0.1700

Data Index: 243807 (Open Web Text)

Max Activating Token Index: 202

Text #12

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6584. Min Act: -0.1700

Data Index: 7302507 (Open Web Text)

Max Activating Token Index: 612

Text #13

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6502. Min Act: -0.1700

Data Index: 2168450 (Open Web Text)

Max Activating Token Index: 347

Text #14

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6447. Min Act: -0.1700

Data Index: 327068 (Open Web Text)

Max Activating Token Index: 188

Text #15

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6393. Min Act: -0.1700

Data Index: 2776667 (Open Web Text)

Max Activating Token Index: 113

Text #16

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6434. Min Act: -0.1637

Data Index: 5129156 (Open Web Text)

Max Activating Token Index: 476

Text #17

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6492. Min Act: -0.1700

Data Index: 1341807 (Open Web Text)

Max Activating Token Index: 95

Text #18

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6463. Min Act: -0.1698

Data Index: 547671 (Open Web Text)

Max Activating Token Index: 420

Text #19

Max Range: 1.7066. Min Range: -1.7066

Max Act: 1.6448. Min Act: -0.1700

Data Index: 1807629 (Open Web Text)

Max Activating Token Index: 323