Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 653 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.2999. Min Act: -0.1700

Data Index: 3655628 (Open Web Text)

Max Activating Token Index: 33

Text #1

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.2416. Min Act: -0.1700

Data Index: 6232834 (Open Web Text)

Max Activating Token Index: 13

Text #2

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.2808. Min Act: -0.1700

Data Index: 1363148 (Open Web Text)

Max Activating Token Index: 311

Text #3

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1686. Min Act: -0.1700

Data Index: 1293274 (Open Web Text)

Max Activating Token Index: 655

Text #4

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1568. Min Act: -0.1700

Data Index: 4098851 (Open Web Text)

Max Activating Token Index: 991

Text #5

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1746. Min Act: -0.1700

Data Index: 5000962 (Open Web Text)

Max Activating Token Index: 15

Text #6

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1478. Min Act: -0.1700

Data Index: 7364117 (Open Web Text)

Max Activating Token Index: 280

Text #7

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1617. Min Act: -0.1700

Data Index: 5252169 (Open Web Text)

Max Activating Token Index: 1000

Text #8

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1506. Min Act: -0.1700

Data Index: 8283230 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1347. Min Act: -0.1700

Data Index: 3401948 (Open Web Text)

Max Activating Token Index: 56

Text #10

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1152. Min Act: -0.1700

Data Index: 2461813 (Open Web Text)

Max Activating Token Index: 334

Text #11

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0805. Min Act: -0.1700

Data Index: 3488650 (Open Web Text)

Max Activating Token Index: 171

Text #12

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0965. Min Act: -0.1700

Data Index: 6601419 (Open Web Text)

Max Activating Token Index: 504

Text #13

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1067. Min Act: -0.1700

Data Index: 402612 (Open Web Text)

Max Activating Token Index: 640

Text #14

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1188. Min Act: -0.1700

Data Index: 171837 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.1021. Min Act: -0.1700

Data Index: 1741505 (Open Web Text)

Max Activating Token Index: 513

Text #16

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0860. Min Act: -0.1700

Data Index: 3122680 (Open Web Text)

Max Activating Token Index: 190

Text #17

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0948. Min Act: -0.1700

Data Index: 8312110 (Open Web Text)

Max Activating Token Index: 674

Text #18

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0860. Min Act: -0.1700

Data Index: 5647706 (Open Web Text)

Max Activating Token Index: 416

Text #19

Max Range: 3.2999. Min Range: -3.2999

Max Act: 3.0855. Min Act: -0.1700

Data Index: 626413 (Open Web Text)

Max Activating Token Index: 831