Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 978 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.3379. Min Act: -0.1700

Data Index: 2425129 (Open Web Text)

Max Activating Token Index: 602

Text #1

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.3152. Min Act: -0.1700

Data Index: 1003776 (Open Web Text)

Max Activating Token Index: 812

Text #2

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.2414. Min Act: -0.1700

Data Index: 4252514 (Open Web Text)

Max Activating Token Index: 557

Text #3

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.2028. Min Act: -0.1700

Data Index: 4156819 (Open Web Text)

Max Activating Token Index: 932

Text #4

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.2126. Min Act: -0.1700

Data Index: 3730694 (Open Web Text)

Max Activating Token Index: 945

Text #5

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1948. Min Act: -0.1700

Data Index: 793712 (Open Web Text)

Max Activating Token Index: 724

Text #6

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1996. Min Act: -0.1700

Data Index: 8410779 (Open Web Text)

Max Activating Token Index: 932

Text #7

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1974. Min Act: -0.1700

Data Index: 5423423 (Open Web Text)

Max Activating Token Index: 463

Text #8

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.2090. Min Act: -0.1700

Data Index: 4590783 (Open Web Text)

Max Activating Token Index: 269

Text #9

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1840. Min Act: -0.1700

Data Index: 6597216 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1832. Min Act: -0.1700

Data Index: 3619650 (Open Web Text)

Max Activating Token Index: 900

Text #11

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.2028. Min Act: -0.1700

Data Index: 116730 (Open Web Text)

Max Activating Token Index: 497

Text #12

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1982. Min Act: -0.1700

Data Index: 8509339 (Open Web Text)

Max Activating Token Index: 646

Text #13

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1977. Min Act: -0.1700

Data Index: 3195892 (Open Web Text)

Max Activating Token Index: 88

Text #14

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1771. Min Act: -0.1700

Data Index: 7423486 (Open Web Text)

Max Activating Token Index: 109

Text #15

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1802. Min Act: -0.1700

Data Index: 6206815 (Open Web Text)

Max Activating Token Index: 211

Text #16

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1616. Min Act: -0.1700

Data Index: 4435702 (Open Web Text)

Max Activating Token Index: 478

Text #17

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1816. Min Act: -0.1700

Data Index: 1779302 (Open Web Text)

Max Activating Token Index: 398

Text #18

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1762. Min Act: -0.1700

Data Index: 5773288 (Open Web Text)

Max Activating Token Index: 644

Text #19

Max Range: 2.3379. Min Range: -2.3379

Max Act: 2.1723. Min Act: -0.1700

Data Index: 2398853 (Open Web Text)

Max Activating Token Index: 4