Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2387 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5680. Min Act: -0.1653

Data Index: 245448 (Open Web Text)

Max Activating Token Index: 14

Text #1

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5262. Min Act: -0.1602

Data Index: 4891045 (Open Web Text)

Max Activating Token Index: 9

Text #2

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5255. Min Act: -0.1692

Data Index: 5062489 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5271. Min Act: -0.1623

Data Index: 2560706 (Open Web Text)

Max Activating Token Index: 18

Text #4

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5196. Min Act: -0.1695

Data Index: 8474359 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.5054. Min Act: -0.1682

Data Index: 3090893 (Open Web Text)

Max Activating Token Index: 16

Text #6

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4962. Min Act: -0.1679

Data Index: 7328046 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4962. Min Act: -0.1166

Data Index: 1861774 (Open Web Text)

Max Activating Token Index: 25

Text #8

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4934. Min Act: -0.1690

Data Index: 5875875 (Open Web Text)

Max Activating Token Index: 28

Text #9

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4898. Min Act: -0.1700

Data Index: 5483414 (Open Web Text)

Max Activating Token Index: 16

Text #10

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4960. Min Act: -0.1663

Data Index: 245449 (Open Web Text)

Max Activating Token Index: 11

Text #11

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4804. Min Act: -0.1699

Data Index: 959261 (Open Web Text)

Max Activating Token Index: 14

Text #12

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4912. Min Act: -0.1698

Data Index: 611486 (Open Web Text)

Max Activating Token Index: 23

Text #13

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4697. Min Act: -0.1683

Data Index: 4207903 (Open Web Text)

Max Activating Token Index: 17

Text #14

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4807. Min Act: -0.1700

Data Index: 698270 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4644. Min Act: -0.1695

Data Index: 8803799 (Open Web Text)

Max Activating Token Index: 34

Text #16

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4652. Min Act: -0.1595

Data Index: 7175439 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4639. Min Act: -0.1566

Data Index: 1621845 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4577. Min Act: -0.1696

Data Index: 7746228 (Open Web Text)

Max Activating Token Index: 41

Text #19

Max Range: 1.5680. Min Range: -1.5680

Max Act: 1.4506. Min Act: -0.1644

Data Index: 8057621 (Open Web Text)

Max Activating Token Index: 14