Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2928 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7694. Min Act: -0.1700

Data Index: 7736691 (Open Web Text)

Max Activating Token Index: 729

Text #1

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7985. Min Act: -0.1700

Data Index: 4508895 (Open Web Text)

Max Activating Token Index: 278

Text #2

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7793. Min Act: -0.1700

Data Index: 6951452 (Open Web Text)

Max Activating Token Index: 542

Text #3

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.8223. Min Act: -0.1700

Data Index: 4797778 (Open Web Text)

Max Activating Token Index: 572

Text #4

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7840. Min Act: -0.1700

Data Index: 3838015 (Open Web Text)

Max Activating Token Index: 529

Text #5

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7536. Min Act: -0.1700

Data Index: 6369148 (Open Web Text)

Max Activating Token Index: 448

Text #6

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7624. Min Act: -0.1700

Data Index: 2260082 (Open Web Text)

Max Activating Token Index: 160

Text #7

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7841. Min Act: -0.1700

Data Index: 524356 (Open Web Text)

Max Activating Token Index: 937

Text #8

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7452. Min Act: -0.1700

Data Index: 1544899 (Open Web Text)

Max Activating Token Index: 458

Text #9

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7421. Min Act: -0.1700

Data Index: 2599530 (Open Web Text)

Max Activating Token Index: 121

Text #10

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7557. Min Act: -0.1700

Data Index: 1390922 (Open Web Text)

Max Activating Token Index: 523

Text #11

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7564. Min Act: -0.1700

Data Index: 2298883 (Open Web Text)

Max Activating Token Index: 865

Text #12

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7567. Min Act: -0.1700

Data Index: 3785430 (Open Web Text)

Max Activating Token Index: 927

Text #13

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7478. Min Act: -0.1700

Data Index: 5304123 (Open Web Text)

Max Activating Token Index: 603

Text #14

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7204. Min Act: -0.1700

Data Index: 3415552 (Open Web Text)

Max Activating Token Index: 490

Text #15

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7046. Min Act: -0.1700

Data Index: 3988957 (Open Web Text)

Max Activating Token Index: 406

Text #16

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7138. Min Act: -0.1700

Data Index: 3231856 (Open Web Text)

Max Activating Token Index: 40

Text #17

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.7078. Min Act: -0.1700

Data Index: 4007822 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.6953. Min Act: -0.1700

Data Index: 571842 (Open Web Text)

Max Activating Token Index: 440

Text #19

Max Range: 3.8223. Min Range: -3.8223

Max Act: 3.6850. Min Act: -0.1700

Data Index: 7496182 (Open Web Text)

Max Activating Token Index: 853