Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1101 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6867. Min Act: -0.1700

Data Index: 4991746 (Open Web Text)

Max Activating Token Index: 305

Text #1

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6976. Min Act: -0.1700

Data Index: 8662126 (Open Web Text)

Max Activating Token Index: 107

Text #2

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6658. Min Act: -0.1700

Data Index: 5524502 (Open Web Text)

Max Activating Token Index: 957

Text #3

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6285. Min Act: -0.1700

Data Index: 7087965 (Open Web Text)

Max Activating Token Index: 376

Text #4

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6392. Min Act: -0.1700

Data Index: 3028483 (Open Web Text)

Max Activating Token Index: 660

Text #5

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6261. Min Act: -0.1700

Data Index: 8062328 (Open Web Text)

Max Activating Token Index: 947

Text #6

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6391. Min Act: -0.1700

Data Index: 3030852 (Open Web Text)

Max Activating Token Index: 678

Text #7

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5767. Min Act: -0.1700

Data Index: 4408031 (Open Web Text)

Max Activating Token Index: 970

Text #8

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5737. Min Act: -0.1700

Data Index: 5492364 (Open Web Text)

Max Activating Token Index: 995

Text #9

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5687. Min Act: -0.1700

Data Index: 4198111 (Open Web Text)

Max Activating Token Index: 971

Text #10

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5812. Min Act: -0.1700

Data Index: 2484174 (Open Web Text)

Max Activating Token Index: 485

Text #11

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6077. Min Act: -0.1700

Data Index: 6016667 (Open Web Text)

Max Activating Token Index: 1002

Text #12

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.6001. Min Act: -0.1700

Data Index: 657759 (Open Web Text)

Max Activating Token Index: 525

Text #13

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5849. Min Act: -0.1700

Data Index: 6686083 (Open Web Text)

Max Activating Token Index: 144

Text #14

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5575. Min Act: -0.1700

Data Index: 3094989 (Open Web Text)

Max Activating Token Index: 675

Text #15

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5436. Min Act: -0.1700

Data Index: 3734687 (Open Web Text)

Max Activating Token Index: 627

Text #16

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5581. Min Act: -0.1700

Data Index: 8674118 (Open Web Text)

Max Activating Token Index: 46

Text #17

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5428. Min Act: -0.1700

Data Index: 4405487 (Open Web Text)

Max Activating Token Index: 977

Text #18

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5702. Min Act: -0.1700

Data Index: 4509558 (Open Web Text)

Max Activating Token Index: 503

Text #19

Max Range: 3.6976. Min Range: -3.6976

Max Act: 3.5459. Min Act: -0.1700

Data Index: 404794 (Open Web Text)

Max Activating Token Index: 195