Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2497 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9751. Min Act: -0.1700

Data Index: 850180 (Open Web Text)

Max Activating Token Index: 415

Text #1

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9493. Min Act: -0.1700

Data Index: 4629190 (Open Web Text)

Max Activating Token Index: 403

Text #2

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8778. Min Act: -0.1700

Data Index: 4712032 (Open Web Text)

Max Activating Token Index: 146

Text #3

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9000. Min Act: -0.1700

Data Index: 5704466 (Open Web Text)

Max Activating Token Index: 85

Text #4

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9203. Min Act: -0.1700

Data Index: 4372333 (Open Web Text)

Max Activating Token Index: 238

Text #5

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9146. Min Act: -0.1700

Data Index: 8329154 (Open Web Text)

Max Activating Token Index: 69

Text #6

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9240. Min Act: -0.1700

Data Index: 3254404 (Open Web Text)

Max Activating Token Index: 440

Text #7

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9095. Min Act: -0.1700

Data Index: 6213285 (Open Web Text)

Max Activating Token Index: 221

Text #8

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8989. Min Act: -0.1700

Data Index: 6653136 (Open Web Text)

Max Activating Token Index: 484

Text #9

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9254. Min Act: -0.1700

Data Index: 300483 (Open Web Text)

Max Activating Token Index: 391

Text #10

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9001. Min Act: -0.1700

Data Index: 6966215 (Open Web Text)

Max Activating Token Index: 204

Text #11

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8794. Min Act: -0.1700

Data Index: 4377899 (Open Web Text)

Max Activating Token Index: 303

Text #12

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9268. Min Act: -0.1700

Data Index: 579104 (Open Web Text)

Max Activating Token Index: 563

Text #13

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8598. Min Act: -0.1700

Data Index: 8299384 (Open Web Text)

Max Activating Token Index: 493

Text #14

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8918. Min Act: -0.1700

Data Index: 3952132 (Open Web Text)

Max Activating Token Index: 445

Text #15

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8790. Min Act: -0.1700

Data Index: 6544255 (Open Web Text)

Max Activating Token Index: 681

Text #16

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9023. Min Act: -0.1700

Data Index: 1400868 (Open Web Text)

Max Activating Token Index: 552

Text #17

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8985. Min Act: -0.1700

Data Index: 1437051 (Open Web Text)

Max Activating Token Index: 531

Text #18

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.9294. Min Act: -0.1700

Data Index: 5904716 (Open Web Text)

Max Activating Token Index: 862

Text #19

Max Range: 4.9751. Min Range: -4.9751

Max Act: 4.8905. Min Act: -0.1700

Data Index: 5440516 (Open Web Text)

Max Activating Token Index: 4