Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2709 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.5228. Min Act: -0.1700

Data Index: 3818086 (Open Web Text)

Max Activating Token Index: 928

Text #1

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.5109. Min Act: -0.1700

Data Index: 1226839 (Open Web Text)

Max Activating Token Index: 482

Text #2

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4980. Min Act: -0.1700

Data Index: 7758307 (Open Web Text)

Max Activating Token Index: 376

Text #3

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.5003. Min Act: -0.1700

Data Index: 895065 (Open Web Text)

Max Activating Token Index: 561

Text #4

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4901. Min Act: -0.1700

Data Index: 2240221 (Open Web Text)

Max Activating Token Index: 285

Text #5

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4868. Min Act: -0.1700

Data Index: 7897716 (Open Web Text)

Max Activating Token Index: 947

Text #6

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4747. Min Act: -0.1700

Data Index: 8445938 (Open Web Text)

Max Activating Token Index: 999

Text #7

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4685. Min Act: -0.1700

Data Index: 2659611 (Open Web Text)

Max Activating Token Index: 437

Text #8

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4687. Min Act: -0.1700

Data Index: 1428744 (Open Web Text)

Max Activating Token Index: 108

Text #9

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4400. Min Act: -0.1700

Data Index: 8530642 (Open Web Text)

Max Activating Token Index: 203

Text #10

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4424. Min Act: -0.1700

Data Index: 4835364 (Open Web Text)

Max Activating Token Index: 508

Text #11

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4567. Min Act: -0.1700

Data Index: 758443 (Open Web Text)

Max Activating Token Index: 336

Text #12

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4203. Min Act: -0.1700

Data Index: 2647038 (Open Web Text)

Max Activating Token Index: 296

Text #13

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4274. Min Act: -0.1700

Data Index: 5052387 (Open Web Text)

Max Activating Token Index: 722

Text #14

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4043. Min Act: -0.1700

Data Index: 3028225 (Open Web Text)

Max Activating Token Index: 373

Text #15

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4222. Min Act: -0.1700

Data Index: 1055492 (Open Web Text)

Max Activating Token Index: 364

Text #16

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4116. Min Act: -0.1700

Data Index: 144966 (Open Web Text)

Max Activating Token Index: 30

Text #17

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4163. Min Act: -0.1700

Data Index: 4103543 (Open Web Text)

Max Activating Token Index: 820

Text #18

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.4127. Min Act: -0.1700

Data Index: 1010024 (Open Web Text)

Max Activating Token Index: 877

Text #19

Max Range: 4.5228. Min Range: -4.5228

Max Act: 4.3657. Min Act: -0.1700

Data Index: 268720 (Open Web Text)

Max Activating Token Index: 836