Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 590 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.7501. Min Act: -0.1700

Data Index: 5407965 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6586. Min Act: -0.1700

Data Index: 5526702 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6107. Min Act: -0.1700

Data Index: 6442784 (Open Web Text)

Max Activating Token Index: 554

Text #3

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6280. Min Act: -0.1700

Data Index: 3222095 (Open Web Text)

Max Activating Token Index: 262

Text #4

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5617. Min Act: -0.1700

Data Index: 7700499 (Open Web Text)

Max Activating Token Index: 763

Text #5

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6385. Min Act: -0.1700

Data Index: 6734771 (Open Web Text)

Max Activating Token Index: 741

Text #6

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6483. Min Act: -0.1700

Data Index: 6944178 (Open Web Text)

Max Activating Token Index: 388

Text #7

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6257. Min Act: -0.1700

Data Index: 530219 (Open Web Text)

Max Activating Token Index: 367

Text #8

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6153. Min Act: -0.1700

Data Index: 7494329 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6230. Min Act: -0.1700

Data Index: 5787826 (Open Web Text)

Max Activating Token Index: 520

Text #10

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6017. Min Act: -0.1700

Data Index: 4985983 (Open Web Text)

Max Activating Token Index: 234

Text #11

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.6057. Min Act: -0.1700

Data Index: 3544668 (Open Web Text)

Max Activating Token Index: 703

Text #12

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5857. Min Act: -0.1700

Data Index: 2044892 (Open Web Text)

Max Activating Token Index: 499

Text #13

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5806. Min Act: -0.1700

Data Index: 8449732 (Open Web Text)

Max Activating Token Index: 20

Text #14

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5747. Min Act: -0.1700

Data Index: 2771308 (Open Web Text)

Max Activating Token Index: 817

Text #15

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5918. Min Act: -0.1700

Data Index: 3385943 (Open Web Text)

Max Activating Token Index: 509

Text #16

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5676. Min Act: -0.1700

Data Index: 8366398 (Open Web Text)

Max Activating Token Index: 63

Text #17

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5993. Min Act: -0.1700

Data Index: 5752720 (Open Web Text)

Max Activating Token Index: 539

Text #18

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5807. Min Act: -0.1700

Data Index: 2254714 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 1.7501. Min Range: -1.7501

Max Act: 1.5726. Min Act: -0.1700

Data Index: 6610112 (Open Web Text)

Max Activating Token Index: 201