Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 47 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1373. Min Range: -4.1373

Max Act: 4.1373. Min Act: -0.1700

Data Index: 2983987 (Open Web Text)

Max Activating Token Index: 655

Text #1

Max Range: 4.1373. Min Range: -4.1373

Max Act: 4.0731. Min Act: -0.1700

Data Index: 5527720 (Open Web Text)

Max Activating Token Index: 915

Text #2

Max Range: 4.1373. Min Range: -4.1373

Max Act: 4.0787. Min Act: -0.1700

Data Index: 2355769 (Open Web Text)

Max Activating Token Index: 578

Text #3

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9349. Min Act: -0.1700

Data Index: 1701348 (Open Web Text)

Max Activating Token Index: 614

Text #4

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9571. Min Act: -0.1700

Data Index: 8231623 (Open Web Text)

Max Activating Token Index: 501

Text #5

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9685. Min Act: -0.1700

Data Index: 5454278 (Open Web Text)

Max Activating Token Index: 387

Text #6

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9160. Min Act: -0.1700

Data Index: 2331650 (Open Web Text)

Max Activating Token Index: 389

Text #7

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9588. Min Act: -0.1700

Data Index: 7660292 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9268. Min Act: -0.1700

Data Index: 5796081 (Open Web Text)

Max Activating Token Index: 560

Text #9

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9139. Min Act: -0.1700

Data Index: 2316508 (Open Web Text)

Max Activating Token Index: 462

Text #10

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.8883. Min Act: -0.1700

Data Index: 5664566 (Open Web Text)

Max Activating Token Index: 375

Text #11

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9142. Min Act: -0.1700

Data Index: 1573560 (Open Web Text)

Max Activating Token Index: 950

Text #12

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.8975. Min Act: -0.1700

Data Index: 5947971 (Open Web Text)

Max Activating Token Index: 260

Text #13

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.8849. Min Act: -0.1700

Data Index: 6023595 (Open Web Text)

Max Activating Token Index: 54

Text #14

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9027. Min Act: -0.1700

Data Index: 6232800 (Open Web Text)

Max Activating Token Index: 696

Text #15

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9258. Min Act: -0.1700

Data Index: 7253209 (Open Web Text)

Max Activating Token Index: 965

Text #16

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9000. Min Act: -0.1700

Data Index: 990860 (Open Web Text)

Max Activating Token Index: 514

Text #17

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.8804. Min Act: -0.1700

Data Index: 5265291 (Open Web Text)

Max Activating Token Index: 237

Text #18

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9029. Min Act: -0.1700

Data Index: 2940531 (Open Web Text)

Max Activating Token Index: 871

Text #19

Max Range: 4.1373. Min Range: -4.1373

Max Act: 3.9025. Min Act: -0.1700

Data Index: 5565595 (Open Web Text)

Max Activating Token Index: 118