Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2605 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6744. Min Act: -0.1700

Data Index: 4887628 (Open Web Text)

Max Activating Token Index: 382

Text #1

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6501. Min Act: -0.1700

Data Index: 3698006 (Open Web Text)

Max Activating Token Index: 97

Text #2

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6522. Min Act: -0.1700

Data Index: 6476432 (Open Web Text)

Max Activating Token Index: 601

Text #3

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6171. Min Act: -0.1700

Data Index: 5779872 (Open Web Text)

Max Activating Token Index: 455

Text #4

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6091. Min Act: -0.1700

Data Index: 5947197 (Open Web Text)

Max Activating Token Index: 298

Text #5

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.6036. Min Act: -0.1700

Data Index: 1102007 (Open Web Text)

Max Activating Token Index: 18

Text #6

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5983. Min Act: -0.1700

Data Index: 7635792 (Open Web Text)

Max Activating Token Index: 143

Text #7

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5995. Min Act: -0.1700

Data Index: 5150070 (Open Web Text)

Max Activating Token Index: 249

Text #8

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5844. Min Act: -0.1700

Data Index: 1326221 (Open Web Text)

Max Activating Token Index: 340

Text #9

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5879. Min Act: -0.1700

Data Index: 7037494 (Open Web Text)

Max Activating Token Index: 655

Text #10

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5892. Min Act: -0.1700

Data Index: 5662780 (Open Web Text)

Max Activating Token Index: 694

Text #11

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5733. Min Act: -0.1700

Data Index: 2554332 (Open Web Text)

Max Activating Token Index: 384

Text #12

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5775. Min Act: -0.1700

Data Index: 8642743 (Open Web Text)

Max Activating Token Index: 415

Text #13

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5763. Min Act: -0.1700

Data Index: 7305878 (Open Web Text)

Max Activating Token Index: 376

Text #14

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5619. Min Act: -0.1700

Data Index: 7146406 (Open Web Text)

Max Activating Token Index: 610

Text #15

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5694. Min Act: -0.1700

Data Index: 1371582 (Open Web Text)

Max Activating Token Index: 827

Text #16

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5748. Min Act: -0.1700

Data Index: 4430313 (Open Web Text)

Max Activating Token Index: 718

Text #17

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5681. Min Act: -0.1700

Data Index: 2941461 (Open Web Text)

Max Activating Token Index: 463

Text #18

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5599. Min Act: -0.1700

Data Index: 5585147 (Open Web Text)

Max Activating Token Index: 348

Text #19

Max Range: 1.6744. Min Range: -1.6744

Max Act: 1.5635. Min Act: -0.1700

Data Index: 1330571 (Open Web Text)

Max Activating Token Index: 559