Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1340 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.6083. Min Act: -0.1700

Data Index: 2638106 (Open Web Text)

Max Activating Token Index: 51

Text #1

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5850. Min Act: -0.1700

Data Index: 6096433 (Open Web Text)

Max Activating Token Index: 883

Text #2

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5433. Min Act: -0.1700

Data Index: 6060975 (Open Web Text)

Max Activating Token Index: 604

Text #3

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5403. Min Act: -0.1700

Data Index: 2915082 (Open Web Text)

Max Activating Token Index: 121

Text #4

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5303. Min Act: -0.1700

Data Index: 2014317 (Open Web Text)

Max Activating Token Index: 122

Text #5

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5236. Min Act: -0.1700

Data Index: 6158044 (Open Web Text)

Max Activating Token Index: 314

Text #6

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5232. Min Act: -0.1700

Data Index: 2395162 (Open Web Text)

Max Activating Token Index: 103

Text #7

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5130. Min Act: -0.1700

Data Index: 98766 (Open Web Text)

Max Activating Token Index: 162

Text #8

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5113. Min Act: -0.1700

Data Index: 1554447 (Open Web Text)

Max Activating Token Index: 877

Text #9

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5195. Min Act: -0.1700

Data Index: 924996 (Open Web Text)

Max Activating Token Index: 67

Text #10

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5193. Min Act: -0.1700

Data Index: 7079449 (Open Web Text)

Max Activating Token Index: 115

Text #11

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5104. Min Act: -0.1700

Data Index: 3961741 (Open Web Text)

Max Activating Token Index: 912

Text #12

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5149. Min Act: -0.1698

Data Index: 5457890 (Open Web Text)

Max Activating Token Index: 63

Text #13

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5081. Min Act: -0.1700

Data Index: 5795518 (Open Web Text)

Max Activating Token Index: 364

Text #14

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5014. Min Act: -0.1700

Data Index: 5035930 (Open Web Text)

Max Activating Token Index: 410

Text #15

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5078. Min Act: -0.1696

Data Index: 802231 (Open Web Text)

Max Activating Token Index: 105

Text #16

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5087. Min Act: -0.1695

Data Index: 838548 (Open Web Text)

Max Activating Token Index: 494

Text #17

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.5132. Min Act: -0.1700

Data Index: 4382794 (Open Web Text)

Max Activating Token Index: 384

Text #18

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.4990. Min Act: -0.1694

Data Index: 6382741 (Open Web Text)

Max Activating Token Index: 54

Text #19

Max Range: 1.6083. Min Range: -1.6083

Max Act: 1.4991. Min Act: -0.1700

Data Index: 3920213 (Open Web Text)

Max Activating Token Index: 452