Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 946 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6396. Min Act: -0.1700

Data Index: 4421526 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6554. Min Act: -0.1700

Data Index: 1472296 (Open Web Text)

Max Activating Token Index: 241

Text #2

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6149. Min Act: -0.1700

Data Index: 7226600 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6184. Min Act: -0.1700

Data Index: 7613281 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6043. Min Act: -0.1700

Data Index: 2450411 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6027. Min Act: -0.1700

Data Index: 636008 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5979. Min Act: -0.1700

Data Index: 889120 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5847. Min Act: -0.1700

Data Index: 3639062 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5848. Min Act: -0.1700

Data Index: 3481949 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5923. Min Act: -0.1700

Data Index: 8764487 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5983. Min Act: -0.1700

Data Index: 8744193 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5743. Min Act: -0.1700

Data Index: 8664914 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5751. Min Act: -0.1700

Data Index: 8405446 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5772. Min Act: -0.1700

Data Index: 1262787 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.6013. Min Act: -0.1700

Data Index: 4352787 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5711. Min Act: -0.1700

Data Index: 4164016 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5848. Min Act: -0.1700

Data Index: 8308688 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5687. Min Act: -0.1700

Data Index: 2740939 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5396. Min Act: -0.1700

Data Index: 3243108 (Open Web Text)

Max Activating Token Index: 13

Text #19

Max Range: 3.6554. Min Range: -3.6554

Max Act: 3.5801. Min Act: -0.1700

Data Index: 4417291 (Open Web Text)

Max Activating Token Index: 5