Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2165 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0863. Min Act: -0.1700

Data Index: 1836142 (Open Web Text)

Max Activating Token Index: 393

Text #1

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0860. Min Act: -0.1700

Data Index: 8503296 (Open Web Text)

Max Activating Token Index: 203

Text #2

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0517. Min Act: -0.1700

Data Index: 4216981 (Open Web Text)

Max Activating Token Index: 849

Text #3

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0096. Min Act: -0.1700

Data Index: 6725610 (Open Web Text)

Max Activating Token Index: 13

Text #4

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9962. Min Act: -0.1700

Data Index: 5147149 (Open Web Text)

Max Activating Token Index: 51

Text #5

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9686. Min Act: -0.1700

Data Index: 4334304 (Open Web Text)

Max Activating Token Index: 394

Text #6

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9869. Min Act: -0.1700

Data Index: 3977503 (Open Web Text)

Max Activating Token Index: 601

Text #7

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0046. Min Act: -0.1700

Data Index: 6298579 (Open Web Text)

Max Activating Token Index: 821

Text #8

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9979. Min Act: -0.1700

Data Index: 2358716 (Open Web Text)

Max Activating Token Index: 89

Text #9

Max Range: 4.0863. Min Range: -4.0863

Max Act: 4.0085. Min Act: -0.1700

Data Index: 2950896 (Open Web Text)

Max Activating Token Index: 76

Text #10

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9756. Min Act: -0.1700

Data Index: 3095351 (Open Web Text)

Max Activating Token Index: 13

Text #11

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9640. Min Act: -0.1700

Data Index: 1888622 (Open Web Text)

Max Activating Token Index: 664

Text #12

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9508. Min Act: -0.1700

Data Index: 7668483 (Open Web Text)

Max Activating Token Index: 929

Text #13

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9112. Min Act: -0.1700

Data Index: 5657639 (Open Web Text)

Max Activating Token Index: 885

Text #14

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9291. Min Act: -0.1700

Data Index: 662772 (Open Web Text)

Max Activating Token Index: 141

Text #15

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9560. Min Act: -0.1700

Data Index: 5107441 (Open Web Text)

Max Activating Token Index: 347

Text #16

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9164. Min Act: -0.1700

Data Index: 390505 (Open Web Text)

Max Activating Token Index: 232

Text #17

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9197. Min Act: -0.1700

Data Index: 8013570 (Open Web Text)

Max Activating Token Index: 503

Text #18

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.8901. Min Act: -0.1700

Data Index: 1567230 (Open Web Text)

Max Activating Token Index: 282

Text #19

Max Range: 4.0863. Min Range: -4.0863

Max Act: 3.9220. Min Act: -0.1700

Data Index: 7763690 (Open Web Text)

Max Activating Token Index: 642