Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 584 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.3974. Min Act: -0.1700

Data Index: 1839722 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.3418. Min Act: -0.1700

Data Index: 3863247 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2979. Min Act: -0.1700

Data Index: 4907806 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2855. Min Act: -0.1700

Data Index: 7149382 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2563. Min Act: -0.1700

Data Index: 2201932 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2623. Min Act: -0.1700

Data Index: 7828662 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2563. Min Act: -0.1700

Data Index: 6631676 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.2150. Min Act: -0.1700

Data Index: 2479214 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1671. Min Act: -0.1700

Data Index: 8821957 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1668. Min Act: -0.1700

Data Index: 7752000 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1559. Min Act: -0.1700

Data Index: 7619415 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1817. Min Act: -0.1700

Data Index: 5735422 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1812. Min Act: -0.1700

Data Index: 8644204 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1869. Min Act: -0.1700

Data Index: 8173642 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1697. Min Act: -0.1700

Data Index: 1023785 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1550. Min Act: -0.1700

Data Index: 7769097 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1342. Min Act: -0.1700

Data Index: 4262667 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1311. Min Act: -0.1700

Data Index: 2863528 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1339. Min Act: -0.1700

Data Index: 2983021 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 6.3974. Min Range: -6.3974

Max Act: 6.1546. Min Act: -0.1700

Data Index: 4681174 (Open Web Text)

Max Activating Token Index: 4