Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2031 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.7511. Min Act: -0.1700

Data Index: 3750114 (Open Web Text)

Max Activating Token Index: 199

Text #1

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5906. Min Act: -0.1700

Data Index: 1159403 (Open Web Text)

Max Activating Token Index: 165

Text #2

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5629. Min Act: -0.1700

Data Index: 7857430 (Open Web Text)

Max Activating Token Index: 891

Text #3

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5972. Min Act: -0.1700

Data Index: 8790069 (Open Web Text)

Max Activating Token Index: 647

Text #4

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5730. Min Act: -0.1700

Data Index: 7790533 (Open Web Text)

Max Activating Token Index: 726

Text #5

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5662. Min Act: -0.1700

Data Index: 8273932 (Open Web Text)

Max Activating Token Index: 456

Text #6

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5158. Min Act: -0.1700

Data Index: 6566725 (Open Web Text)

Max Activating Token Index: 386

Text #7

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5408. Min Act: -0.1700

Data Index: 4631727 (Open Web Text)

Max Activating Token Index: 210

Text #8

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5042. Min Act: -0.1700

Data Index: 4463095 (Open Web Text)

Max Activating Token Index: 962

Text #9

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5011. Min Act: -0.1700

Data Index: 4860487 (Open Web Text)

Max Activating Token Index: 356

Text #10

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4898. Min Act: -0.1700

Data Index: 459210 (Open Web Text)

Max Activating Token Index: 539

Text #11

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5024. Min Act: -0.1700

Data Index: 3249452 (Open Web Text)

Max Activating Token Index: 228

Text #12

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4924. Min Act: -0.1700

Data Index: 3104584 (Open Web Text)

Max Activating Token Index: 48

Text #13

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.5082. Min Act: -0.1700

Data Index: 8169807 (Open Web Text)

Max Activating Token Index: 40

Text #14

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4882. Min Act: -0.1700

Data Index: 8049633 (Open Web Text)

Max Activating Token Index: 637

Text #15

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4602. Min Act: -0.1700

Data Index: 7896154 (Open Web Text)

Max Activating Token Index: 540

Text #16

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4939. Min Act: -0.1700

Data Index: 7775075 (Open Web Text)

Max Activating Token Index: 58

Text #17

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4611. Min Act: -0.1700

Data Index: 6903730 (Open Web Text)

Max Activating Token Index: 475

Text #18

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4792. Min Act: -0.1700

Data Index: 7136161 (Open Web Text)

Max Activating Token Index: 573

Text #19

Max Range: 5.7511. Min Range: -5.7511

Max Act: 5.4505. Min Act: -0.1700

Data Index: 7140056 (Open Web Text)

Max Activating Token Index: 700