Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 754 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1048. Min Range: -5.1048

Max Act: 5.1048. Min Act: -0.1700

Data Index: 826710 (Open Web Text)

Max Activating Token Index: 53

Text #1

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.9206. Min Act: -0.1700

Data Index: 1068853 (Open Web Text)

Max Activating Token Index: 469

Text #2

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.9705. Min Act: -0.1700

Data Index: 861691 (Open Web Text)

Max Activating Token Index: 176

Text #3

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8926. Min Act: -0.1700

Data Index: 6153726 (Open Web Text)

Max Activating Token Index: 637

Text #4

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8545. Min Act: -0.1700

Data Index: 3145841 (Open Web Text)

Max Activating Token Index: 741

Text #5

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8718. Min Act: -0.1700

Data Index: 4477767 (Open Web Text)

Max Activating Token Index: 268

Text #6

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8622. Min Act: -0.1700

Data Index: 5923218 (Open Web Text)

Max Activating Token Index: 108

Text #7

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8207. Min Act: -0.1700

Data Index: 3386926 (Open Web Text)

Max Activating Token Index: 64

Text #8

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8453. Min Act: -0.1700

Data Index: 2038944 (Open Web Text)

Max Activating Token Index: 404

Text #9

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.8252. Min Act: -0.1700

Data Index: 7975620 (Open Web Text)

Max Activating Token Index: 48

Text #10

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7758. Min Act: -0.1700

Data Index: 4372382 (Open Web Text)

Max Activating Token Index: 200

Text #11

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7046. Min Act: -0.1700

Data Index: 388240 (Open Web Text)

Max Activating Token Index: 496

Text #12

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7080. Min Act: -0.1700

Data Index: 7874544 (Open Web Text)

Max Activating Token Index: 643

Text #13

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7179. Min Act: -0.1700

Data Index: 826708 (Open Web Text)

Max Activating Token Index: 77

Text #14

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7168. Min Act: -0.1700

Data Index: 4358577 (Open Web Text)

Max Activating Token Index: 843

Text #15

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7209. Min Act: -0.1700

Data Index: 7679339 (Open Web Text)

Max Activating Token Index: 987

Text #16

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7114. Min Act: -0.1700

Data Index: 7679340 (Open Web Text)

Max Activating Token Index: 169

Text #17

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.7041. Min Act: -0.1700

Data Index: 8667426 (Open Web Text)

Max Activating Token Index: 362

Text #18

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.6341. Min Act: -0.1700

Data Index: 4699572 (Open Web Text)

Max Activating Token Index: 171

Text #19

Max Range: 5.1048. Min Range: -5.1048

Max Act: 4.6193. Min Act: -0.1700

Data Index: 5496598 (Open Web Text)

Max Activating Token Index: 682