Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 611 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.3760. Min Act: -0.1700

Data Index: 1140723 (Open Web Text)

Max Activating Token Index: 542

Text #1

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.3566. Min Act: -0.1700

Data Index: 1976218 (Open Web Text)

Max Activating Token Index: 627

Text #2

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.3226. Min Act: -0.1700

Data Index: 5065874 (Open Web Text)

Max Activating Token Index: 624

Text #3

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.3271. Min Act: -0.1700

Data Index: 3980339 (Open Web Text)

Max Activating Token Index: 256

Text #4

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2892. Min Act: -0.1700

Data Index: 493323 (Open Web Text)

Max Activating Token Index: 284

Text #5

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2869. Min Act: -0.1700

Data Index: 3553647 (Open Web Text)

Max Activating Token Index: 490

Text #6

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2978. Min Act: -0.1700

Data Index: 2495985 (Open Web Text)

Max Activating Token Index: 411

Text #7

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2696. Min Act: -0.1700

Data Index: 4190037 (Open Web Text)

Max Activating Token Index: 608

Text #8

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2252. Min Act: -0.1700

Data Index: 4583231 (Open Web Text)

Max Activating Token Index: 723

Text #9

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2198. Min Act: -0.1700

Data Index: 5514731 (Open Web Text)

Max Activating Token Index: 635

Text #10

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2338. Min Act: -0.1700

Data Index: 824640 (Open Web Text)

Max Activating Token Index: 355

Text #11

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2470. Min Act: -0.1700

Data Index: 6290801 (Open Web Text)

Max Activating Token Index: 843

Text #12

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.2037. Min Act: -0.1700

Data Index: 860604 (Open Web Text)

Max Activating Token Index: 96

Text #13

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1910. Min Act: -0.1700

Data Index: 3348810 (Open Web Text)

Max Activating Token Index: 473

Text #14

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1823. Min Act: -0.1700

Data Index: 3835086 (Open Web Text)

Max Activating Token Index: 345

Text #15

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1898. Min Act: -0.1700

Data Index: 7764336 (Open Web Text)

Max Activating Token Index: 379

Text #16

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1491. Min Act: -0.1700

Data Index: 1103741 (Open Web Text)

Max Activating Token Index: 342

Text #17

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1751. Min Act: -0.1700

Data Index: 5943404 (Open Web Text)

Max Activating Token Index: 515

Text #18

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1642. Min Act: -0.1700

Data Index: 999960 (Open Web Text)

Max Activating Token Index: 472

Text #19

Max Range: 4.3760. Min Range: -4.3760

Max Act: 4.1814. Min Act: -0.1700

Data Index: 4139040 (Open Web Text)

Max Activating Token Index: 703