Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 61 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.2108. Min Act: -0.1700

Data Index: 5350708 (Open Web Text)

Max Activating Token Index: 271

Text #1

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.1583. Min Act: -0.1700

Data Index: 7649334 (Open Web Text)

Max Activating Token Index: 577

Text #2

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.1341. Min Act: -0.1700

Data Index: 6055538 (Open Web Text)

Max Activating Token Index: 975

Text #3

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.0775. Min Act: -0.1700

Data Index: 3653677 (Open Web Text)

Max Activating Token Index: 677

Text #4

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.0705. Min Act: -0.1700

Data Index: 8218071 (Open Web Text)

Max Activating Token Index: 938

Text #5

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.0604. Min Act: -0.1700

Data Index: 3695625 (Open Web Text)

Max Activating Token Index: 884

Text #6

Max Range: 3.2108. Min Range: -3.2108

Max Act: 3.0176. Min Act: -0.1700

Data Index: 6442615 (Open Web Text)

Max Activating Token Index: 926

Text #7

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9896. Min Act: -0.1700

Data Index: 1318910 (Open Web Text)

Max Activating Token Index: 986

Text #8

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9982. Min Act: -0.1700

Data Index: 6298355 (Open Web Text)

Max Activating Token Index: 371

Text #9

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9705. Min Act: -0.1700

Data Index: 509480 (Open Web Text)

Max Activating Token Index: 622

Text #10

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9163. Min Act: -0.1700

Data Index: 7243648 (Open Web Text)

Max Activating Token Index: 614

Text #11

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9274. Min Act: -0.1700

Data Index: 3032936 (Open Web Text)

Max Activating Token Index: 980

Text #12

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9317. Min Act: -0.1700

Data Index: 1264948 (Open Web Text)

Max Activating Token Index: 532

Text #13

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9289. Min Act: -0.1700

Data Index: 3143980 (Open Web Text)

Max Activating Token Index: 592

Text #14

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9192. Min Act: -0.1700

Data Index: 5823894 (Open Web Text)

Max Activating Token Index: 565

Text #15

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9229. Min Act: -0.1700

Data Index: 3313358 (Open Web Text)

Max Activating Token Index: 680

Text #16

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9380. Min Act: -0.1700

Data Index: 3195340 (Open Web Text)

Max Activating Token Index: 577

Text #17

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.8799. Min Act: -0.1700

Data Index: 3735484 (Open Web Text)

Max Activating Token Index: 743

Text #18

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.8857. Min Act: -0.1700

Data Index: 1571223 (Open Web Text)

Max Activating Token Index: 803

Text #19

Max Range: 3.2108. Min Range: -3.2108

Max Act: 2.9104. Min Act: -0.1700

Data Index: 2908450 (Open Web Text)

Max Activating Token Index: 872