Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2775 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.7282. Min Act: -0.1700

Data Index: 1466542 (Open Web Text)

Max Activating Token Index: 487

Text #1

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.6924. Min Act: -0.1700

Data Index: 3995993 (Open Web Text)

Max Activating Token Index: 80

Text #2

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.5089. Min Act: -0.1700

Data Index: 506649 (Open Web Text)

Max Activating Token Index: 519

Text #3

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.4632. Min Act: -0.1700

Data Index: 4066247 (Open Web Text)

Max Activating Token Index: 751

Text #4

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.4817. Min Act: -0.1700

Data Index: 506644 (Open Web Text)

Max Activating Token Index: 290

Text #5

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.4535. Min Act: -0.1700

Data Index: 6002903 (Open Web Text)

Max Activating Token Index: 412

Text #6

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.4409. Min Act: -0.1700

Data Index: 506647 (Open Web Text)

Max Activating Token Index: 80

Text #7

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3182. Min Act: -0.1700

Data Index: 2827612 (Open Web Text)

Max Activating Token Index: 152

Text #8

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3371. Min Act: -0.1700

Data Index: 184894 (Open Web Text)

Max Activating Token Index: 534

Text #9

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3388. Min Act: -0.1700

Data Index: 506651 (Open Web Text)

Max Activating Token Index: 267

Text #10

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3283. Min Act: -0.1700

Data Index: 7554987 (Open Web Text)

Max Activating Token Index: 571

Text #11

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2908. Min Act: -0.1700

Data Index: 1009946 (Open Web Text)

Max Activating Token Index: 448

Text #12

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3167. Min Act: -0.1700

Data Index: 506634 (Open Web Text)

Max Activating Token Index: 252

Text #13

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2814. Min Act: -0.1700

Data Index: 38946 (Open Web Text)

Max Activating Token Index: 474

Text #14

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.3119. Min Act: -0.1700

Data Index: 1009944 (Open Web Text)

Max Activating Token Index: 444

Text #15

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2799. Min Act: -0.1700

Data Index: 1009942 (Open Web Text)

Max Activating Token Index: 415

Text #16

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2705. Min Act: -0.1700

Data Index: 5532453 (Open Web Text)

Max Activating Token Index: 923

Text #17

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2617. Min Act: -0.1700

Data Index: 506636 (Open Web Text)

Max Activating Token Index: 374

Text #18

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2700. Min Act: -0.1700

Data Index: 744611 (Open Web Text)

Max Activating Token Index: 336

Text #19

Max Range: 5.7282. Min Range: -5.7282

Max Act: 5.2772. Min Act: -0.1700

Data Index: 4368713 (Open Web Text)

Max Activating Token Index: 366