Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2956 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.6651. Min Act: -0.1700

Data Index: 5885709 (Open Web Text)

Max Activating Token Index: 376

Text #1

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.5933. Min Act: -0.1700

Data Index: 2653136 (Open Web Text)

Max Activating Token Index: 512

Text #2

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.5943. Min Act: -0.1700

Data Index: 6782017 (Open Web Text)

Max Activating Token Index: 693

Text #3

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.5246. Min Act: -0.1700

Data Index: 1954722 (Open Web Text)

Max Activating Token Index: 392

Text #4

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4875. Min Act: -0.1700

Data Index: 4502769 (Open Web Text)

Max Activating Token Index: 451

Text #5

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4753. Min Act: -0.1700

Data Index: 3729824 (Open Web Text)

Max Activating Token Index: 940

Text #6

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.5098. Min Act: -0.1700

Data Index: 7920211 (Open Web Text)

Max Activating Token Index: 728

Text #7

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4746. Min Act: -0.1700

Data Index: 4309780 (Open Web Text)

Max Activating Token Index: 382

Text #8

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4733. Min Act: -0.1700

Data Index: 1953430 (Open Web Text)

Max Activating Token Index: 637

Text #9

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4391. Min Act: -0.1700

Data Index: 3867980 (Open Web Text)

Max Activating Token Index: 639

Text #10

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4494. Min Act: -0.1700

Data Index: 6635125 (Open Web Text)

Max Activating Token Index: 130

Text #11

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4294. Min Act: -0.1700

Data Index: 8430799 (Open Web Text)

Max Activating Token Index: 760

Text #12

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4270. Min Act: -0.1700

Data Index: 489352 (Open Web Text)

Max Activating Token Index: 703

Text #13

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4217. Min Act: -0.1700

Data Index: 7751090 (Open Web Text)

Max Activating Token Index: 737

Text #14

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4467. Min Act: -0.1700

Data Index: 6737977 (Open Web Text)

Max Activating Token Index: 881

Text #15

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4003. Min Act: -0.1700

Data Index: 2070097 (Open Web Text)

Max Activating Token Index: 391

Text #16

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4236. Min Act: -0.1700

Data Index: 4923186 (Open Web Text)

Max Activating Token Index: 907

Text #17

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4076. Min Act: -0.1700

Data Index: 2130628 (Open Web Text)

Max Activating Token Index: 757

Text #18

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.3939. Min Act: -0.1700

Data Index: 1122892 (Open Web Text)

Max Activating Token Index: 791

Text #19

Max Range: 3.6651. Min Range: -3.6651

Max Act: 3.4079. Min Act: -0.1700

Data Index: 6530784 (Open Web Text)

Max Activating Token Index: 546