Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1209 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.5139. Min Act: -0.1700

Data Index: 8010711 (Open Web Text)

Max Activating Token Index: 410

Text #1

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.4447. Min Act: -0.1700

Data Index: 2906514 (Open Web Text)

Max Activating Token Index: 395

Text #2

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.4677. Min Act: -0.1700

Data Index: 868360 (Open Web Text)

Max Activating Token Index: 194

Text #3

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.3870. Min Act: -0.1700

Data Index: 5593241 (Open Web Text)

Max Activating Token Index: 818

Text #4

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.3491. Min Act: -0.1700

Data Index: 6279194 (Open Web Text)

Max Activating Token Index: 244

Text #5

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.3219. Min Act: -0.1700

Data Index: 1938083 (Open Web Text)

Max Activating Token Index: 105

Text #6

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.3186. Min Act: -0.1700

Data Index: 2575314 (Open Web Text)

Max Activating Token Index: 838

Text #7

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2983. Min Act: -0.1700

Data Index: 7886654 (Open Web Text)

Max Activating Token Index: 261

Text #8

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2841. Min Act: -0.1700

Data Index: 8216275 (Open Web Text)

Max Activating Token Index: 637

Text #9

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2951. Min Act: -0.1700

Data Index: 5441496 (Open Web Text)

Max Activating Token Index: 586

Text #10

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2899. Min Act: -0.1700

Data Index: 2175675 (Open Web Text)

Max Activating Token Index: 176

Text #11

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2526. Min Act: -0.1700

Data Index: 7299537 (Open Web Text)

Max Activating Token Index: 605

Text #12

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2686. Min Act: -0.1700

Data Index: 8794213 (Open Web Text)

Max Activating Token Index: 238

Text #13

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2275. Min Act: -0.1700

Data Index: 5356739 (Open Web Text)

Max Activating Token Index: 917

Text #14

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2221. Min Act: -0.1700

Data Index: 6932363 (Open Web Text)

Max Activating Token Index: 865

Text #15

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2073. Min Act: -0.1700

Data Index: 7863904 (Open Web Text)

Max Activating Token Index: 846

Text #16

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.2222. Min Act: -0.1700

Data Index: 3046727 (Open Web Text)

Max Activating Token Index: 506

Text #17

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.1986. Min Act: -0.1700

Data Index: 8486800 (Open Web Text)

Max Activating Token Index: 883

Text #18

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.1722. Min Act: -0.1700

Data Index: 3763807 (Open Web Text)

Max Activating Token Index: 464

Text #19

Max Range: 4.5139. Min Range: -4.5139

Max Act: 4.1811. Min Act: -0.1700

Data Index: 2308442 (Open Web Text)

Max Activating Token Index: 345