Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1543 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.2564. Min Act: -0.1700

Data Index: 6045201 (Open Web Text)

Max Activating Token Index: 16

Text #1

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.2094. Min Act: -0.1700

Data Index: 3590688 (Open Web Text)

Max Activating Token Index: 37

Text #2

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1989. Min Act: -0.1700

Data Index: 1115116 (Open Web Text)

Max Activating Token Index: 346

Text #3

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1978. Min Act: -0.1700

Data Index: 1199505 (Open Web Text)

Max Activating Token Index: 695

Text #4

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1946. Min Act: -0.1700

Data Index: 7367803 (Open Web Text)

Max Activating Token Index: 102

Text #5

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1726. Min Act: -0.1700

Data Index: 8246503 (Open Web Text)

Max Activating Token Index: 791

Text #6

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1452. Min Act: -0.1700

Data Index: 1436466 (Open Web Text)

Max Activating Token Index: 49

Text #7

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0837. Min Act: -0.1700

Data Index: 5474439 (Open Web Text)

Max Activating Token Index: 332

Text #8

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1124. Min Act: -0.1700

Data Index: 8806360 (Open Web Text)

Max Activating Token Index: 921

Text #9

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0984. Min Act: -0.1700

Data Index: 2257016 (Open Web Text)

Max Activating Token Index: 806

Text #10

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0864. Min Act: -0.1700

Data Index: 6334961 (Open Web Text)

Max Activating Token Index: 547

Text #11

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.1157. Min Act: -0.1700

Data Index: 474972 (Open Web Text)

Max Activating Token Index: 563

Text #12

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0986. Min Act: -0.1700

Data Index: 5654765 (Open Web Text)

Max Activating Token Index: 257

Text #13

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0777. Min Act: -0.1700

Data Index: 6838960 (Open Web Text)

Max Activating Token Index: 268

Text #14

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0610. Min Act: -0.1700

Data Index: 4047105 (Open Web Text)

Max Activating Token Index: 296

Text #15

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0977. Min Act: -0.1700

Data Index: 6576532 (Open Web Text)

Max Activating Token Index: 328

Text #16

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0859. Min Act: -0.1700

Data Index: 3547240 (Open Web Text)

Max Activating Token Index: 521

Text #17

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0659. Min Act: -0.1700

Data Index: 6282374 (Open Web Text)

Max Activating Token Index: 43

Text #18

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0619. Min Act: -0.1700

Data Index: 6470910 (Open Web Text)

Max Activating Token Index: 32

Text #19

Max Range: 3.2564. Min Range: -3.2564

Max Act: 3.0714. Min Act: -0.1700

Data Index: 6467947 (Open Web Text)

Max Activating Token Index: 395