Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2713 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.7642. Min Act: -0.1700

Data Index: 3834833 (Open Web Text)

Max Activating Token Index: 663

Text #1

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.5943. Min Act: -0.1700

Data Index: 5679552 (Open Web Text)

Max Activating Token Index: 462

Text #2

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.5132. Min Act: -0.1700

Data Index: 6646507 (Open Web Text)

Max Activating Token Index: 792

Text #3

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.5385. Min Act: -0.1700

Data Index: 2264024 (Open Web Text)

Max Activating Token Index: 595

Text #4

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4509. Min Act: -0.1700

Data Index: 2689467 (Open Web Text)

Max Activating Token Index: 435

Text #5

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.5138. Min Act: -0.1700

Data Index: 5123772 (Open Web Text)

Max Activating Token Index: 721

Text #6

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4601. Min Act: -0.1700

Data Index: 238844 (Open Web Text)

Max Activating Token Index: 413

Text #7

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4355. Min Act: -0.1700

Data Index: 6644674 (Open Web Text)

Max Activating Token Index: 478

Text #8

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4862. Min Act: -0.1700

Data Index: 5974530 (Open Web Text)

Max Activating Token Index: 284

Text #9

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4836. Min Act: -0.1700

Data Index: 6648199 (Open Web Text)

Max Activating Token Index: 807

Text #10

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4864. Min Act: -0.1700

Data Index: 6818702 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4530. Min Act: -0.1700

Data Index: 8304941 (Open Web Text)

Max Activating Token Index: 659

Text #12

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4180. Min Act: -0.1700

Data Index: 1496488 (Open Web Text)

Max Activating Token Index: 379

Text #13

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4236. Min Act: -0.1700

Data Index: 1585697 (Open Web Text)

Max Activating Token Index: 651

Text #14

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.3973. Min Act: -0.1700

Data Index: 3529642 (Open Web Text)

Max Activating Token Index: 20

Text #15

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4183. Min Act: -0.1700

Data Index: 281318 (Open Web Text)

Max Activating Token Index: 683

Text #16

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.3317. Min Act: -0.1700

Data Index: 5874339 (Open Web Text)

Max Activating Token Index: 580

Text #17

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4401. Min Act: -0.1700

Data Index: 8553826 (Open Web Text)

Max Activating Token Index: 379

Text #18

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.4077. Min Act: -0.1700

Data Index: 7245971 (Open Web Text)

Max Activating Token Index: 880

Text #19

Max Range: 5.7642. Min Range: -5.7642

Max Act: 5.3839. Min Act: -0.1700

Data Index: 5345306 (Open Web Text)

Max Activating Token Index: 871