Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1450 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.9737. Min Act: -0.1686

Data Index: 3462581 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.9227. Min Act: -0.1700

Data Index: 3562011 (Open Web Text)

Max Activating Token Index: 14

Text #2

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.9071. Min Act: -0.1688

Data Index: 2722668 (Open Web Text)

Max Activating Token Index: 9

Text #3

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8992. Min Act: -0.1700

Data Index: 6682522 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8808. Min Act: -0.1700

Data Index: 3927460 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8888. Min Act: -0.1695

Data Index: 3740957 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8734. Min Act: -0.1686

Data Index: 1043813 (Open Web Text)

Max Activating Token Index: 10

Text #7

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8726. Min Act: -0.1700

Data Index: 2858661 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8763. Min Act: -0.1699

Data Index: 3097608 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8729. Min Act: -0.1692

Data Index: 7582861 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8707. Min Act: -0.1699

Data Index: 6268480 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8633. Min Act: -0.1699

Data Index: 2636062 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8656. Min Act: -0.1669

Data Index: 4183 (Open Web Text)

Max Activating Token Index: 15

Text #13

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8549. Min Act: -0.1648

Data Index: 8564221 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8576. Min Act: -0.1691

Data Index: 2200893 (Open Web Text)

Max Activating Token Index: 9

Text #15

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8503. Min Act: -0.1631

Data Index: 4447234 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8558. Min Act: -0.1697

Data Index: 8342563 (Open Web Text)

Max Activating Token Index: 7

Text #17

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8556. Min Act: -0.1698

Data Index: 8312351 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8447. Min Act: -0.1700

Data Index: 7441530 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 1.9737. Min Range: -1.9737

Max Act: 1.8513. Min Act: -0.1698

Data Index: 5128462 (Open Web Text)

Max Activating Token Index: 4