Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1559 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.4204. Min Act: -0.1700

Data Index: 3708447 (Open Web Text)

Max Activating Token Index: 885

Text #1

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.4141. Min Act: -0.1700

Data Index: 9133 (Open Web Text)

Max Activating Token Index: 367

Text #2

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.4188. Min Act: -0.1700

Data Index: 3917447 (Open Web Text)

Max Activating Token Index: 602

Text #3

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.4264. Min Act: -0.1700

Data Index: 6824922 (Open Web Text)

Max Activating Token Index: 190

Text #4

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.4017. Min Act: -0.1700

Data Index: 842217 (Open Web Text)

Max Activating Token Index: 180

Text #5

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3886. Min Act: -0.1700

Data Index: 304574 (Open Web Text)

Max Activating Token Index: 175

Text #6

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3825. Min Act: -0.1700

Data Index: 977198 (Open Web Text)

Max Activating Token Index: 380

Text #7

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3917. Min Act: -0.1700

Data Index: 7303463 (Open Web Text)

Max Activating Token Index: 179

Text #8

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3973. Min Act: -0.1700

Data Index: 2066992 (Open Web Text)

Max Activating Token Index: 474

Text #9

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3857. Min Act: -0.1700

Data Index: 4349251 (Open Web Text)

Max Activating Token Index: 815

Text #10

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3874. Min Act: -0.1700

Data Index: 4681413 (Open Web Text)

Max Activating Token Index: 56

Text #11

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3848. Min Act: -0.1700

Data Index: 3293787 (Open Web Text)

Max Activating Token Index: 183

Text #12

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3661. Min Act: -0.1700

Data Index: 3404977 (Open Web Text)

Max Activating Token Index: 370

Text #13

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3764. Min Act: -0.1700

Data Index: 3364942 (Open Web Text)

Max Activating Token Index: 319

Text #14

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3755. Min Act: -0.1700

Data Index: 5463170 (Open Web Text)

Max Activating Token Index: 900

Text #15

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3808. Min Act: -0.1700

Data Index: 4840887 (Open Web Text)

Max Activating Token Index: 832

Text #16

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3667. Min Act: -0.1700

Data Index: 2864816 (Open Web Text)

Max Activating Token Index: 618

Text #17

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3583. Min Act: -0.1700

Data Index: 629280 (Open Web Text)

Max Activating Token Index: 201

Text #18

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3688. Min Act: -0.1700

Data Index: 99327 (Open Web Text)

Max Activating Token Index: 477

Text #19

Max Range: 3.4264. Min Range: -3.4264

Max Act: 3.3748. Min Act: -0.1700

Data Index: 1158829 (Open Web Text)

Max Activating Token Index: 191