Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1278 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.4758. Min Act: -0.1700

Data Index: 7918545 (Open Web Text)

Max Activating Token Index: 358

Text #1

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.5298. Min Act: -0.1700

Data Index: 8654305 (Open Web Text)

Max Activating Token Index: 994

Text #2

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.5098. Min Act: -0.1700

Data Index: 3472651 (Open Web Text)

Max Activating Token Index: 298

Text #3

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.4076. Min Act: -0.1700

Data Index: 3456573 (Open Web Text)

Max Activating Token Index: 488

Text #4

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.3436. Min Act: -0.1700

Data Index: 2240194 (Open Web Text)

Max Activating Token Index: 874

Text #5

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2111. Min Act: -0.1700

Data Index: 6599693 (Open Web Text)

Max Activating Token Index: 821

Text #6

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2521. Min Act: -0.1700

Data Index: 4562893 (Open Web Text)

Max Activating Token Index: 993

Text #7

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2452. Min Act: -0.1700

Data Index: 141351 (Open Web Text)

Max Activating Token Index: 435

Text #8

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2671. Min Act: -0.1700

Data Index: 1749608 (Open Web Text)

Max Activating Token Index: 464

Text #9

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2479. Min Act: -0.1700

Data Index: 8655110 (Open Web Text)

Max Activating Token Index: 340

Text #10

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2423. Min Act: -0.1700

Data Index: 6542575 (Open Web Text)

Max Activating Token Index: 829

Text #11

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2035. Min Act: -0.1700

Data Index: 2469489 (Open Web Text)

Max Activating Token Index: 435

Text #12

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.1993. Min Act: -0.1700

Data Index: 514384 (Open Web Text)

Max Activating Token Index: 874

Text #13

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.2438. Min Act: -0.1700

Data Index: 5521539 (Open Web Text)

Max Activating Token Index: 415

Text #14

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.1886. Min Act: -0.1700

Data Index: 4562894 (Open Web Text)

Max Activating Token Index: 541

Text #15

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.1102. Min Act: -0.1700

Data Index: 1285794 (Open Web Text)

Max Activating Token Index: 444

Text #16

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.1115. Min Act: -0.1700

Data Index: 8374123 (Open Web Text)

Max Activating Token Index: 462

Text #17

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.0988. Min Act: -0.1700

Data Index: 4688494 (Open Web Text)

Max Activating Token Index: 626

Text #18

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.0997. Min Act: -0.1700

Data Index: 5239044 (Open Web Text)

Max Activating Token Index: 534

Text #19

Max Range: 3.5298. Min Range: -3.5298

Max Act: 3.0970. Min Act: -0.1700

Data Index: 1610418 (Open Web Text)

Max Activating Token Index: 179