Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2332 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0362. Min Range: -3.0362

Max Act: 3.0362. Min Act: -0.1700

Data Index: 1066312 (Open Web Text)

Max Activating Token Index: 944

Text #1

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.9419. Min Act: -0.1700

Data Index: 1202254 (Open Web Text)

Max Activating Token Index: 778

Text #2

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.9060. Min Act: -0.1700

Data Index: 347545 (Open Web Text)

Max Activating Token Index: 27

Text #3

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8992. Min Act: -0.1700

Data Index: 4907501 (Open Web Text)

Max Activating Token Index: 1003

Text #4

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.9161. Min Act: -0.1700

Data Index: 3295543 (Open Web Text)

Max Activating Token Index: 449

Text #5

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8567. Min Act: -0.1700

Data Index: 7319041 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8689. Min Act: -0.1700

Data Index: 8785616 (Open Web Text)

Max Activating Token Index: 754

Text #7

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8561. Min Act: -0.1700

Data Index: 6474143 (Open Web Text)

Max Activating Token Index: 878

Text #8

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8636. Min Act: -0.1700

Data Index: 7507193 (Open Web Text)

Max Activating Token Index: 188

Text #9

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8368. Min Act: -0.1700

Data Index: 1471732 (Open Web Text)

Max Activating Token Index: 620

Text #10

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8315. Min Act: -0.1700

Data Index: 8741849 (Open Web Text)

Max Activating Token Index: 900

Text #11

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8409. Min Act: -0.1700

Data Index: 4128398 (Open Web Text)

Max Activating Token Index: 341

Text #12

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8381. Min Act: -0.1700

Data Index: 5998896 (Open Web Text)

Max Activating Token Index: 927

Text #13

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8252. Min Act: -0.1700

Data Index: 2637419 (Open Web Text)

Max Activating Token Index: 99

Text #14

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8423. Min Act: -0.1700

Data Index: 1167556 (Open Web Text)

Max Activating Token Index: 576

Text #15

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8399. Min Act: -0.1700

Data Index: 7844859 (Open Web Text)

Max Activating Token Index: 105

Text #16

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8137. Min Act: -0.1700

Data Index: 3251776 (Open Web Text)

Max Activating Token Index: 312

Text #17

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8428. Min Act: -0.1700

Data Index: 2467455 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8243. Min Act: -0.1700

Data Index: 5632717 (Open Web Text)

Max Activating Token Index: 966

Text #19

Max Range: 3.0362. Min Range: -3.0362

Max Act: 2.8046. Min Act: -0.1700

Data Index: 6564854 (Open Web Text)

Max Activating Token Index: 295