Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1920 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.3569. Min Act: -0.1700

Data Index: 7623541 (Open Web Text)

Max Activating Token Index: 292

Text #1

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.1780. Min Act: -0.1700

Data Index: 694028 (Open Web Text)

Max Activating Token Index: 355

Text #2

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.1485. Min Act: -0.1700

Data Index: 2168573 (Open Web Text)

Max Activating Token Index: 472

Text #3

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.1677. Min Act: -0.1700

Data Index: 1145173 (Open Web Text)

Max Activating Token Index: 531

Text #4

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.1313. Min Act: -0.1700

Data Index: 728811 (Open Web Text)

Max Activating Token Index: 714

Text #5

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0779. Min Act: -0.1700

Data Index: 573968 (Open Web Text)

Max Activating Token Index: 176

Text #6

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0965. Min Act: -0.1700

Data Index: 5193337 (Open Web Text)

Max Activating Token Index: 881

Text #7

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0677. Min Act: -0.1700

Data Index: 8436220 (Open Web Text)

Max Activating Token Index: 596

Text #8

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0910. Min Act: -0.1700

Data Index: 2763024 (Open Web Text)

Max Activating Token Index: 397

Text #9

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0121. Min Act: -0.1700

Data Index: 4430363 (Open Web Text)

Max Activating Token Index: 879

Text #10

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0134. Min Act: -0.1700

Data Index: 3094620 (Open Web Text)

Max Activating Token Index: 345

Text #11

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0400. Min Act: -0.1700

Data Index: 6792484 (Open Web Text)

Max Activating Token Index: 795

Text #12

Max Range: 4.3569. Min Range: -4.3569

Max Act: 4.0079. Min Act: -0.1700

Data Index: 2885850 (Open Web Text)

Max Activating Token Index: 939

Text #13

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9812. Min Act: -0.1700

Data Index: 4126746 (Open Web Text)

Max Activating Token Index: 767

Text #14

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9846. Min Act: -0.1700

Data Index: 6003772 (Open Web Text)

Max Activating Token Index: 431

Text #15

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9936. Min Act: -0.1700

Data Index: 919735 (Open Web Text)

Max Activating Token Index: 628

Text #16

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9994. Min Act: -0.1700

Data Index: 7183831 (Open Web Text)

Max Activating Token Index: 397

Text #17

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9900. Min Act: -0.1700

Data Index: 7337617 (Open Web Text)

Max Activating Token Index: 875

Text #18

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9073. Min Act: -0.1700

Data Index: 3207403 (Open Web Text)

Max Activating Token Index: 989

Text #19

Max Range: 4.3569. Min Range: -4.3569

Max Act: 3.9399. Min Act: -0.1700

Data Index: 1396613 (Open Web Text)

Max Activating Token Index: 558