Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1696 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.7143. Min Act: -0.1700

Data Index: 5241020 (Open Web Text)

Max Activating Token Index: 1008

Text #1

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.6157. Min Act: -0.1700

Data Index: 204890 (Open Web Text)

Max Activating Token Index: 489

Text #2

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.5196. Min Act: -0.1700

Data Index: 6402376 (Open Web Text)

Max Activating Token Index: 504

Text #3

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4688. Min Act: -0.1700

Data Index: 7303650 (Open Web Text)

Max Activating Token Index: 629

Text #4

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4645. Min Act: -0.1700

Data Index: 6358497 (Open Web Text)

Max Activating Token Index: 267

Text #5

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4580. Min Act: -0.1700

Data Index: 5463876 (Open Web Text)

Max Activating Token Index: 913

Text #6

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4866. Min Act: -0.1700

Data Index: 7689670 (Open Web Text)

Max Activating Token Index: 218

Text #7

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4881. Min Act: -0.1700

Data Index: 3971942 (Open Web Text)

Max Activating Token Index: 751

Text #8

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4486. Min Act: -0.1700

Data Index: 4883284 (Open Web Text)

Max Activating Token Index: 856

Text #9

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4817. Min Act: -0.1700

Data Index: 2903726 (Open Web Text)

Max Activating Token Index: 473

Text #10

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4286. Min Act: -0.1700

Data Index: 6431073 (Open Web Text)

Max Activating Token Index: 941

Text #11

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4528. Min Act: -0.1700

Data Index: 3441661 (Open Web Text)

Max Activating Token Index: 1008

Text #12

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4629. Min Act: -0.1700

Data Index: 2693091 (Open Web Text)

Max Activating Token Index: 91

Text #13

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4115. Min Act: -0.1700

Data Index: 2388018 (Open Web Text)

Max Activating Token Index: 720

Text #14

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.3840. Min Act: -0.1700

Data Index: 7648028 (Open Web Text)

Max Activating Token Index: 450

Text #15

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4355. Min Act: -0.1700

Data Index: 7748397 (Open Web Text)

Max Activating Token Index: 291

Text #16

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4230. Min Act: -0.1700

Data Index: 8376389 (Open Web Text)

Max Activating Token Index: 33

Text #17

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4258. Min Act: -0.1700

Data Index: 6020534 (Open Web Text)

Max Activating Token Index: 951

Text #18

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.4012. Min Act: -0.1700

Data Index: 5463875 (Open Web Text)

Max Activating Token Index: 412

Text #19

Max Range: 4.7143. Min Range: -4.7143

Max Act: 4.3914. Min Act: -0.1700

Data Index: 656409 (Open Web Text)

Max Activating Token Index: 681