Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 241 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.7059. Min Act: -0.1700

Data Index: 6440351 (Open Web Text)

Max Activating Token Index: 482

Text #1

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6567. Min Act: -0.1700

Data Index: 7861079 (Open Web Text)

Max Activating Token Index: 74

Text #2

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6491. Min Act: -0.1700

Data Index: 4304463 (Open Web Text)

Max Activating Token Index: 27

Text #3

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6309. Min Act: -0.1700

Data Index: 2477234 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6204. Min Act: -0.1700

Data Index: 5559011 (Open Web Text)

Max Activating Token Index: 1006

Text #5

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6104. Min Act: -0.1700

Data Index: 2364934 (Open Web Text)

Max Activating Token Index: 303

Text #6

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6050. Min Act: -0.1700

Data Index: 3333876 (Open Web Text)

Max Activating Token Index: 9

Text #7

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5967. Min Act: -0.1700

Data Index: 2217313 (Open Web Text)

Max Activating Token Index: 432

Text #8

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5919. Min Act: -0.1656

Data Index: 3762577 (Open Web Text)

Max Activating Token Index: 43

Text #9

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.6001. Min Act: -0.1700

Data Index: 8222790 (Open Web Text)

Max Activating Token Index: 661

Text #10

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5813. Min Act: -0.1700

Data Index: 8007508 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5903. Min Act: -0.1694

Data Index: 1952000 (Open Web Text)

Max Activating Token Index: 8

Text #12

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5805. Min Act: -0.1699

Data Index: 3487893 (Open Web Text)

Max Activating Token Index: 631

Text #13

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5748. Min Act: -0.1699

Data Index: 2049289 (Open Web Text)

Max Activating Token Index: 33

Text #14

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5757. Min Act: -0.1699

Data Index: 811392 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5688. Min Act: -0.1699

Data Index: 703392 (Open Web Text)

Max Activating Token Index: 12

Text #16

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5613. Min Act: -0.1700

Data Index: 755041 (Open Web Text)

Max Activating Token Index: 78

Text #17

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5604. Min Act: -0.1700

Data Index: 743672 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5707. Min Act: -0.1700

Data Index: 1310043 (Open Web Text)

Max Activating Token Index: 465

Text #19

Max Range: 1.7059. Min Range: -1.7059

Max Act: 1.5636. Min Act: -0.1700

Data Index: 3943421 (Open Web Text)

Max Activating Token Index: 9