Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1853 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0621. Min Range: -2.0621

Max Act: 2.0621. Min Act: -0.1700

Data Index: 7594488 (Open Web Text)

Max Activating Token Index: 14

Text #1

Max Range: 2.0621. Min Range: -2.0621

Max Act: 2.0251. Min Act: -0.1700

Data Index: 1685494 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 2.0621. Min Range: -2.0621

Max Act: 2.0272. Min Act: -0.1700

Data Index: 6911959 (Open Web Text)

Max Activating Token Index: 20

Text #3

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9957. Min Act: -0.1700

Data Index: 8504690 (Open Web Text)

Max Activating Token Index: 15

Text #4

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9946. Min Act: -0.1700

Data Index: 8132860 (Open Web Text)

Max Activating Token Index: 9

Text #5

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9802. Min Act: -0.1700

Data Index: 6233420 (Open Web Text)

Max Activating Token Index: 29

Text #6

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9805. Min Act: -0.1700

Data Index: 2621048 (Open Web Text)

Max Activating Token Index: 13

Text #7

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9803. Min Act: -0.1700

Data Index: 6274805 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9850. Min Act: -0.1700

Data Index: 1738237 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9848. Min Act: -0.1700

Data Index: 6830191 (Open Web Text)

Max Activating Token Index: 24

Text #10

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9668. Min Act: -0.1700

Data Index: 2038566 (Open Web Text)

Max Activating Token Index: 20

Text #11

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9754. Min Act: -0.1700

Data Index: 5914664 (Open Web Text)

Max Activating Token Index: 9

Text #12

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9772. Min Act: -0.1700

Data Index: 1219053 (Open Web Text)

Max Activating Token Index: 116

Text #13

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9721. Min Act: -0.1700

Data Index: 6900914 (Open Web Text)

Max Activating Token Index: 444

Text #14

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9691. Min Act: -0.1700

Data Index: 5573564 (Open Web Text)

Max Activating Token Index: 61

Text #15

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9681. Min Act: -0.1700

Data Index: 3075702 (Open Web Text)

Max Activating Token Index: 111

Text #16

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9693. Min Act: -0.1700

Data Index: 4231018 (Open Web Text)

Max Activating Token Index: 294

Text #17

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9720. Min Act: -0.1700

Data Index: 6369938 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9720. Min Act: -0.1700

Data Index: 1897421 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 2.0621. Min Range: -2.0621

Max Act: 1.9662. Min Act: -0.1700

Data Index: 2667209 (Open Web Text)

Max Activating Token Index: 33