Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1795 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.9716. Min Act: -0.1700

Data Index: 2877917 (Open Web Text)

Max Activating Token Index: 106

Text #1

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.7583. Min Act: -0.1700

Data Index: 5211852 (Open Web Text)

Max Activating Token Index: 603

Text #2

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.7010. Min Act: -0.1700

Data Index: 153848 (Open Web Text)

Max Activating Token Index: 972

Text #3

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.6733. Min Act: -0.1700

Data Index: 709078 (Open Web Text)

Max Activating Token Index: 384

Text #4

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.7492. Min Act: -0.1700

Data Index: 7961069 (Open Web Text)

Max Activating Token Index: 433

Text #5

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.6155. Min Act: -0.1700

Data Index: 2311513 (Open Web Text)

Max Activating Token Index: 609

Text #6

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.6134. Min Act: -0.1700

Data Index: 5103251 (Open Web Text)

Max Activating Token Index: 347

Text #7

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.5034. Min Act: -0.1700

Data Index: 5886129 (Open Web Text)

Max Activating Token Index: 669

Text #8

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.5608. Min Act: -0.1700

Data Index: 1042023 (Open Web Text)

Max Activating Token Index: 210

Text #9

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4978. Min Act: -0.1700

Data Index: 1569783 (Open Web Text)

Max Activating Token Index: 660

Text #10

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4884. Min Act: -0.1700

Data Index: 919543 (Open Web Text)

Max Activating Token Index: 894

Text #11

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4649. Min Act: -0.1700

Data Index: 7521003 (Open Web Text)

Max Activating Token Index: 758

Text #12

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4360. Min Act: -0.1700

Data Index: 1444900 (Open Web Text)

Max Activating Token Index: 443

Text #13

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4666. Min Act: -0.1700

Data Index: 4754322 (Open Web Text)

Max Activating Token Index: 276

Text #14

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4670. Min Act: -0.1700

Data Index: 8565788 (Open Web Text)

Max Activating Token Index: 549

Text #15

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4012. Min Act: -0.1700

Data Index: 8015362 (Open Web Text)

Max Activating Token Index: 569

Text #16

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4770. Min Act: -0.1700

Data Index: 2706625 (Open Web Text)

Max Activating Token Index: 618

Text #17

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.5288. Min Act: -0.1700

Data Index: 2808546 (Open Web Text)

Max Activating Token Index: 670

Text #18

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4390. Min Act: -0.1700

Data Index: 7711938 (Open Web Text)

Max Activating Token Index: 749

Text #19

Max Range: 5.9716. Min Range: -5.9716

Max Act: 5.4178. Min Act: -0.1700

Data Index: 4950535 (Open Web Text)

Max Activating Token Index: 593