Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2866 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.8536. Min Act: -0.1700

Data Index: 2730629 (Open Web Text)

Max Activating Token Index: 471

Text #1

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.8467. Min Act: -0.1700

Data Index: 624339 (Open Web Text)

Max Activating Token Index: 502

Text #2

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.7332. Min Act: -0.1700

Data Index: 416405 (Open Web Text)

Max Activating Token Index: 590

Text #3

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.7247. Min Act: -0.1700

Data Index: 5297023 (Open Web Text)

Max Activating Token Index: 925

Text #4

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.7344. Min Act: -0.1700

Data Index: 4263140 (Open Web Text)

Max Activating Token Index: 352

Text #5

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.7209. Min Act: -0.1700

Data Index: 7218025 (Open Web Text)

Max Activating Token Index: 161

Text #6

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6767. Min Act: -0.1700

Data Index: 2339233 (Open Web Text)

Max Activating Token Index: 842

Text #7

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6974. Min Act: -0.1700

Data Index: 799459 (Open Web Text)

Max Activating Token Index: 789

Text #8

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.7023. Min Act: -0.1700

Data Index: 2727134 (Open Web Text)

Max Activating Token Index: 115

Text #9

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6836. Min Act: -0.1700

Data Index: 28080 (Open Web Text)

Max Activating Token Index: 880

Text #10

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6870. Min Act: -0.1700

Data Index: 4701101 (Open Web Text)

Max Activating Token Index: 451

Text #11

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6748. Min Act: -0.1700

Data Index: 5486308 (Open Web Text)

Max Activating Token Index: 660

Text #12

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6847. Min Act: -0.1700

Data Index: 1672139 (Open Web Text)

Max Activating Token Index: 549

Text #13

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.5988. Min Act: -0.1700

Data Index: 7262386 (Open Web Text)

Max Activating Token Index: 531

Text #14

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6545. Min Act: -0.1700

Data Index: 2233590 (Open Web Text)

Max Activating Token Index: 715

Text #15

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6327. Min Act: -0.1700

Data Index: 3684206 (Open Web Text)

Max Activating Token Index: 583

Text #16

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6621. Min Act: -0.1700

Data Index: 2625437 (Open Web Text)

Max Activating Token Index: 386

Text #17

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6369. Min Act: -0.1700

Data Index: 4015391 (Open Web Text)

Max Activating Token Index: 1019

Text #18

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6536. Min Act: -0.1700

Data Index: 8007933 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 3.8536. Min Range: -3.8536

Max Act: 3.6316. Min Act: -0.1700

Data Index: 8585281 (Open Web Text)

Max Activating Token Index: 244