Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2931 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.7173. Min Act: -0.1700

Data Index: 2430041 (Open Web Text)

Max Activating Token Index: 71

Text #1

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.6499. Min Act: -0.1700

Data Index: 5698444 (Open Web Text)

Max Activating Token Index: 97

Text #2

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.6346. Min Act: -0.1700

Data Index: 11747 (Open Web Text)

Max Activating Token Index: 74

Text #3

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.6075. Min Act: -0.1700

Data Index: 6457070 (Open Web Text)

Max Activating Token Index: 1000

Text #4

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.6026. Min Act: -0.1700

Data Index: 2445390 (Open Web Text)

Max Activating Token Index: 706

Text #5

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.6059. Min Act: -0.1700

Data Index: 139358 (Open Web Text)

Max Activating Token Index: 16

Text #6

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5972. Min Act: -0.1700

Data Index: 3824652 (Open Web Text)

Max Activating Token Index: 936

Text #7

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5561. Min Act: -0.1700

Data Index: 16603 (Open Web Text)

Max Activating Token Index: 252

Text #8

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5086. Min Act: -0.1700

Data Index: 4312537 (Open Web Text)

Max Activating Token Index: 796

Text #9

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5441. Min Act: -0.1700

Data Index: 4542167 (Open Web Text)

Max Activating Token Index: 707

Text #10

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5375. Min Act: -0.1700

Data Index: 3340824 (Open Web Text)

Max Activating Token Index: 107

Text #11

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4937. Min Act: -0.1700

Data Index: 3560558 (Open Web Text)

Max Activating Token Index: 106

Text #12

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5022. Min Act: -0.1700

Data Index: 2311642 (Open Web Text)

Max Activating Token Index: 504

Text #13

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4847. Min Act: -0.1700

Data Index: 5296099 (Open Web Text)

Max Activating Token Index: 747

Text #14

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.5102. Min Act: -0.1700

Data Index: 8239928 (Open Web Text)

Max Activating Token Index: 38

Text #15

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4865. Min Act: -0.1700

Data Index: 7478241 (Open Web Text)

Max Activating Token Index: 630

Text #16

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4907. Min Act: -0.1700

Data Index: 5488416 (Open Web Text)

Max Activating Token Index: 128

Text #17

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4827. Min Act: -0.1700

Data Index: 3037583 (Open Web Text)

Max Activating Token Index: 46

Text #18

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4893. Min Act: -0.1700

Data Index: 5701814 (Open Web Text)

Max Activating Token Index: 91

Text #19

Max Range: 4.7173. Min Range: -4.7173

Max Act: 4.4942. Min Act: -0.1700

Data Index: 6056718 (Open Web Text)

Max Activating Token Index: 620