Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 100 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5624. Min Act: -0.1700

Data Index: 5901377 (Open Web Text)

Max Activating Token Index: 795

Text #1

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5393. Min Act: -0.1699

Data Index: 8684973 (Open Web Text)

Max Activating Token Index: 864

Text #2

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5386. Min Act: -0.1689

Data Index: 6385488 (Open Web Text)

Max Activating Token Index: 558

Text #3

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5400. Min Act: -0.1697

Data Index: 2916547 (Open Web Text)

Max Activating Token Index: 947

Text #4

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5151. Min Act: -0.1697

Data Index: 7397553 (Open Web Text)

Max Activating Token Index: 617

Text #5

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5219. Min Act: -0.1700

Data Index: 2374262 (Open Web Text)

Max Activating Token Index: 60

Text #6

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5095. Min Act: -0.1689

Data Index: 2584815 (Open Web Text)

Max Activating Token Index: 930

Text #7

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5085. Min Act: -0.1700

Data Index: 1161444 (Open Web Text)

Max Activating Token Index: 709

Text #8

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5108. Min Act: -0.1589

Data Index: 7262684 (Open Web Text)

Max Activating Token Index: 606

Text #9

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5159. Min Act: -0.1681

Data Index: 4344492 (Open Web Text)

Max Activating Token Index: 960

Text #10

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5087. Min Act: -0.1698

Data Index: 3701189 (Open Web Text)

Max Activating Token Index: 97

Text #11

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5120. Min Act: -0.1700

Data Index: 7100312 (Open Web Text)

Max Activating Token Index: 910

Text #12

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5055. Min Act: -0.1700

Data Index: 3650814 (Open Web Text)

Max Activating Token Index: 486

Text #13

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.4974. Min Act: -0.1666

Data Index: 3931865 (Open Web Text)

Max Activating Token Index: 744

Text #14

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.4943. Min Act: -0.1700

Data Index: 2643927 (Open Web Text)

Max Activating Token Index: 646

Text #15

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.4948. Min Act: -0.1698

Data Index: 6231086 (Open Web Text)

Max Activating Token Index: 958

Text #16

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.4980. Min Act: -0.1700

Data Index: 2451752 (Open Web Text)

Max Activating Token Index: 323

Text #17

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5030. Min Act: -0.1679

Data Index: 7313537 (Open Web Text)

Max Activating Token Index: 414

Text #18

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5003. Min Act: -0.1700

Data Index: 1706717 (Open Web Text)

Max Activating Token Index: 583

Text #19

Max Range: 1.5624. Min Range: -1.5624

Max Act: 1.5007. Min Act: -0.1317

Data Index: 4297087 (Open Web Text)

Max Activating Token Index: 421