Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1209 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.6694. Min Act: -0.1700

Data Index: 2844234 (Open Web Text)

Max Activating Token Index: 621

Text #1

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.6018. Min Act: -0.1700

Data Index: 3475516 (Open Web Text)

Max Activating Token Index: 155

Text #2

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5581. Min Act: -0.1700

Data Index: 3446346 (Open Web Text)

Max Activating Token Index: 892

Text #3

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5556. Min Act: -0.1700

Data Index: 3425967 (Open Web Text)

Max Activating Token Index: 346

Text #4

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5345. Min Act: -0.1700

Data Index: 51356 (Open Web Text)

Max Activating Token Index: 116

Text #5

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5217. Min Act: -0.1700

Data Index: 6509139 (Open Web Text)

Max Activating Token Index: 475

Text #6

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5541. Min Act: -0.1700

Data Index: 3750392 (Open Web Text)

Max Activating Token Index: 63

Text #7

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5382. Min Act: -0.1700

Data Index: 2218857 (Open Web Text)

Max Activating Token Index: 31

Text #8

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5301. Min Act: -0.1700

Data Index: 2646280 (Open Web Text)

Max Activating Token Index: 774

Text #9

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5408. Min Act: -0.1700

Data Index: 7119802 (Open Web Text)

Max Activating Token Index: 17

Text #10

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5281. Min Act: -0.1700

Data Index: 504232 (Open Web Text)

Max Activating Token Index: 60

Text #11

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5050. Min Act: -0.1700

Data Index: 4758077 (Open Web Text)

Max Activating Token Index: 701

Text #12

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4973. Min Act: -0.1700

Data Index: 3868700 (Open Web Text)

Max Activating Token Index: 321

Text #13

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4933. Min Act: -0.1700

Data Index: 8432101 (Open Web Text)

Max Activating Token Index: 62

Text #14

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5009. Min Act: -0.1700

Data Index: 1331953 (Open Web Text)

Max Activating Token Index: 170

Text #15

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.5024. Min Act: -0.1700

Data Index: 2214782 (Open Web Text)

Max Activating Token Index: 363

Text #16

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4619. Min Act: -0.1700

Data Index: 4490693 (Open Web Text)

Max Activating Token Index: 684

Text #17

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4911. Min Act: -0.1700

Data Index: 3221077 (Open Web Text)

Max Activating Token Index: 776

Text #18

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4966. Min Act: -0.1700

Data Index: 4692094 (Open Web Text)

Max Activating Token Index: 483

Text #19

Max Range: 4.6694. Min Range: -4.6694

Max Act: 4.4625. Min Act: -0.1700

Data Index: 6324278 (Open Web Text)

Max Activating Token Index: 63