Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1578 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.5225. Min Act: -0.1700

Data Index: 4944129 (Open Web Text)

Max Activating Token Index: 228

Text #1

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.4195. Min Act: -0.1700

Data Index: 5236061 (Open Web Text)

Max Activating Token Index: 361

Text #2

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.4510. Min Act: -0.1700

Data Index: 2179193 (Open Web Text)

Max Activating Token Index: 646

Text #3

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3919. Min Act: -0.1700

Data Index: 3734894 (Open Web Text)

Max Activating Token Index: 865

Text #4

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.4033. Min Act: -0.1700

Data Index: 4772322 (Open Web Text)

Max Activating Token Index: 76

Text #5

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.4054. Min Act: -0.1700

Data Index: 2217150 (Open Web Text)

Max Activating Token Index: 454

Text #6

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3787. Min Act: -0.1700

Data Index: 8701155 (Open Web Text)

Max Activating Token Index: 423

Text #7

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3457. Min Act: -0.1700

Data Index: 3603995 (Open Web Text)

Max Activating Token Index: 22

Text #8

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3571. Min Act: -0.1700

Data Index: 6121761 (Open Web Text)

Max Activating Token Index: 618

Text #9

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3546. Min Act: -0.1700

Data Index: 517076 (Open Web Text)

Max Activating Token Index: 932

Text #10

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3543. Min Act: -0.1700

Data Index: 1243837 (Open Web Text)

Max Activating Token Index: 796

Text #11

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3490. Min Act: -0.1700

Data Index: 3179249 (Open Web Text)

Max Activating Token Index: 323

Text #12

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3390. Min Act: -0.1700

Data Index: 4903429 (Open Web Text)

Max Activating Token Index: 535

Text #13

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3351. Min Act: -0.1700

Data Index: 88538 (Open Web Text)

Max Activating Token Index: 84

Text #14

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3478. Min Act: -0.1700

Data Index: 4253971 (Open Web Text)

Max Activating Token Index: 411

Text #15

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3132. Min Act: -0.1700

Data Index: 6666066 (Open Web Text)

Max Activating Token Index: 112

Text #16

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3187. Min Act: -0.1700

Data Index: 8062793 (Open Web Text)

Max Activating Token Index: 400

Text #17

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3277. Min Act: -0.1700

Data Index: 2011187 (Open Web Text)

Max Activating Token Index: 554

Text #18

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3329. Min Act: -0.1700

Data Index: 4155485 (Open Web Text)

Max Activating Token Index: 447

Text #19

Max Range: 4.5225. Min Range: -4.5225

Max Act: 4.3259. Min Act: -0.1700

Data Index: 1799101 (Open Web Text)

Max Activating Token Index: 899