Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 234 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4178. Min Range: -4.4178

Max Act: 4.4178. Min Act: -0.1700

Data Index: 4349160 (Open Web Text)

Max Activating Token Index: 828

Text #1

Max Range: 4.4178. Min Range: -4.4178

Max Act: 4.1050. Min Act: -0.1700

Data Index: 482988 (Open Web Text)

Max Activating Token Index: 846

Text #2

Max Range: 4.4178. Min Range: -4.4178

Max Act: 4.1449. Min Act: -0.1700

Data Index: 4416891 (Open Web Text)

Max Activating Token Index: 222

Text #3

Max Range: 4.4178. Min Range: -4.4178

Max Act: 4.0448. Min Act: -0.1700

Data Index: 880467 (Open Web Text)

Max Activating Token Index: 777

Text #4

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.9961. Min Act: -0.1700

Data Index: 5820444 (Open Web Text)

Max Activating Token Index: 809

Text #5

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.9824. Min Act: -0.1700

Data Index: 6919857 (Open Web Text)

Max Activating Token Index: 797

Text #6

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.9629. Min Act: -0.1700

Data Index: 4610002 (Open Web Text)

Max Activating Token Index: 834

Text #7

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.8361. Min Act: -0.1700

Data Index: 6330735 (Open Web Text)

Max Activating Token Index: 823

Text #8

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7897. Min Act: -0.1700

Data Index: 3988834 (Open Web Text)

Max Activating Token Index: 252

Text #9

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7656. Min Act: -0.1700

Data Index: 1370733 (Open Web Text)

Max Activating Token Index: 228

Text #10

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7399. Min Act: -0.1700

Data Index: 8036803 (Open Web Text)

Max Activating Token Index: 162

Text #11

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7822. Min Act: -0.1700

Data Index: 5715504 (Open Web Text)

Max Activating Token Index: 728

Text #12

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7473. Min Act: -0.1700

Data Index: 7119836 (Open Web Text)

Max Activating Token Index: 786

Text #13

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7313. Min Act: -0.1700

Data Index: 2864238 (Open Web Text)

Max Activating Token Index: 858

Text #14

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.6757. Min Act: -0.1700

Data Index: 3468303 (Open Web Text)

Max Activating Token Index: 700

Text #15

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7043. Min Act: -0.1700

Data Index: 3670229 (Open Web Text)

Max Activating Token Index: 824

Text #16

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.7092. Min Act: -0.1700

Data Index: 8286822 (Open Web Text)

Max Activating Token Index: 858

Text #17

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.6789. Min Act: -0.1700

Data Index: 4774215 (Open Web Text)

Max Activating Token Index: 807

Text #18

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.6650. Min Act: -0.1700

Data Index: 5362745 (Open Web Text)

Max Activating Token Index: 241

Text #19

Max Range: 4.4178. Min Range: -4.4178

Max Act: 3.6674. Min Act: -0.1700

Data Index: 7491394 (Open Web Text)

Max Activating Token Index: 246