Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2477 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.3020. Min Act: -0.1700

Data Index: 4063363 (Open Web Text)

Max Activating Token Index: 961

Text #1

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.3035. Min Act: -0.1700

Data Index: 8185706 (Open Web Text)

Max Activating Token Index: 140

Text #2

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.2063. Min Act: -0.1700

Data Index: 4694441 (Open Web Text)

Max Activating Token Index: 396

Text #3

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.1698. Min Act: -0.1700

Data Index: 439080 (Open Web Text)

Max Activating Token Index: 211

Text #4

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.1636. Min Act: -0.1700

Data Index: 2485532 (Open Web Text)

Max Activating Token Index: 153

Text #5

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0866. Min Act: -0.1700

Data Index: 7708665 (Open Web Text)

Max Activating Token Index: 595

Text #6

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0702. Min Act: -0.1700

Data Index: 5810618 (Open Web Text)

Max Activating Token Index: 694

Text #7

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0736. Min Act: -0.1700

Data Index: 8003506 (Open Web Text)

Max Activating Token Index: 546

Text #8

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0813. Min Act: -0.1700

Data Index: 1769139 (Open Web Text)

Max Activating Token Index: 217

Text #9

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0567. Min Act: -0.1700

Data Index: 2710762 (Open Web Text)

Max Activating Token Index: 534

Text #10

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0268. Min Act: -0.1700

Data Index: 1696462 (Open Web Text)

Max Activating Token Index: 92

Text #11

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0318. Min Act: -0.1700

Data Index: 7854555 (Open Web Text)

Max Activating Token Index: 188

Text #12

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0156. Min Act: -0.1700

Data Index: 7271400 (Open Web Text)

Max Activating Token Index: 418

Text #13

Max Range: 3.3035. Min Range: -3.3035

Max Act: 3.0134. Min Act: -0.1700

Data Index: 1969775 (Open Web Text)

Max Activating Token Index: 39

Text #14

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9872. Min Act: -0.1700

Data Index: 584449 (Open Web Text)

Max Activating Token Index: 172

Text #15

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9964. Min Act: -0.1700

Data Index: 2728508 (Open Web Text)

Max Activating Token Index: 399

Text #16

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9786. Min Act: -0.1700

Data Index: 7487051 (Open Web Text)

Max Activating Token Index: 115

Text #17

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9726. Min Act: -0.1700

Data Index: 6666139 (Open Web Text)

Max Activating Token Index: 885

Text #18

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9869. Min Act: -0.1700

Data Index: 3756459 (Open Web Text)

Max Activating Token Index: 35

Text #19

Max Range: 3.3035. Min Range: -3.3035

Max Act: 2.9889. Min Act: -0.1700

Data Index: 3763413 (Open Web Text)

Max Activating Token Index: 741