Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2358 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.8300. Min Act: -0.1700

Data Index: 1199066 (Open Web Text)

Max Activating Token Index: 80

Text #1

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.8129. Min Act: -0.1700

Data Index: 4209498 (Open Web Text)

Max Activating Token Index: 701

Text #2

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.8152. Min Act: -0.1700

Data Index: 3847365 (Open Web Text)

Max Activating Token Index: 319

Text #3

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.8247. Min Act: -0.1700

Data Index: 726491 (Open Web Text)

Max Activating Token Index: 655

Text #4

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.8172. Min Act: -0.1700

Data Index: 8235945 (Open Web Text)

Max Activating Token Index: 50

Text #5

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7948. Min Act: -0.1700

Data Index: 3350694 (Open Web Text)

Max Activating Token Index: 249

Text #6

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7861. Min Act: -0.1700

Data Index: 8502123 (Open Web Text)

Max Activating Token Index: 272

Text #7

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7895. Min Act: -0.1700

Data Index: 1781562 (Open Web Text)

Max Activating Token Index: 671

Text #8

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7944. Min Act: -0.1700

Data Index: 7756441 (Open Web Text)

Max Activating Token Index: 737

Text #9

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7845. Min Act: -0.1700

Data Index: 5864152 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7726. Min Act: -0.1700

Data Index: 7957161 (Open Web Text)

Max Activating Token Index: 98

Text #11

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7838. Min Act: -0.1700

Data Index: 829761 (Open Web Text)

Max Activating Token Index: 642

Text #12

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7869. Min Act: -0.1700

Data Index: 7531415 (Open Web Text)

Max Activating Token Index: 741

Text #13

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7738. Min Act: -0.1700

Data Index: 6136699 (Open Web Text)

Max Activating Token Index: 638

Text #14

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7798. Min Act: -0.1700

Data Index: 2528704 (Open Web Text)

Max Activating Token Index: 633

Text #15

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7872. Min Act: -0.1700

Data Index: 91651 (Open Web Text)

Max Activating Token Index: 128

Text #16

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7799. Min Act: -0.1700

Data Index: 5608398 (Open Web Text)

Max Activating Token Index: 25

Text #17

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7528. Min Act: -0.1700

Data Index: 2700143 (Open Web Text)

Max Activating Token Index: 41

Text #18

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7723. Min Act: -0.1700

Data Index: 227507 (Open Web Text)

Max Activating Token Index: 464

Text #19

Max Range: 1.8300. Min Range: -1.8300

Max Act: 1.7599. Min Act: -0.1700

Data Index: 4959086 (Open Web Text)

Max Activating Token Index: 205