Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2359 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.7053. Min Act: -0.1700

Data Index: 4189980 (Open Web Text)

Max Activating Token Index: 817

Text #1

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6849. Min Act: -0.1700

Data Index: 1433088 (Open Web Text)

Max Activating Token Index: 292

Text #2

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6902. Min Act: -0.1700

Data Index: 6419885 (Open Web Text)

Max Activating Token Index: 484

Text #3

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6952. Min Act: -0.1700

Data Index: 2136107 (Open Web Text)

Max Activating Token Index: 52

Text #4

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6553. Min Act: -0.1700

Data Index: 7121131 (Open Web Text)

Max Activating Token Index: 381

Text #5

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6578. Min Act: -0.1700

Data Index: 8246840 (Open Web Text)

Max Activating Token Index: 403

Text #6

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6645. Min Act: -0.1700

Data Index: 3913835 (Open Web Text)

Max Activating Token Index: 200

Text #7

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6070. Min Act: -0.1700

Data Index: 4065260 (Open Web Text)

Max Activating Token Index: 958

Text #8

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6126. Min Act: -0.1700

Data Index: 4179034 (Open Web Text)

Max Activating Token Index: 171

Text #9

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6076. Min Act: -0.1700

Data Index: 8709346 (Open Web Text)

Max Activating Token Index: 511

Text #10

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6202. Min Act: -0.1700

Data Index: 6694543 (Open Web Text)

Max Activating Token Index: 832

Text #11

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.6192. Min Act: -0.1700

Data Index: 6713794 (Open Web Text)

Max Activating Token Index: 348

Text #12

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5864. Min Act: -0.1700

Data Index: 8320288 (Open Web Text)

Max Activating Token Index: 653

Text #13

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5839. Min Act: -0.1700

Data Index: 6704417 (Open Web Text)

Max Activating Token Index: 303

Text #14

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5709. Min Act: -0.1700

Data Index: 3814955 (Open Web Text)

Max Activating Token Index: 301

Text #15

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5660. Min Act: -0.1700

Data Index: 5743331 (Open Web Text)

Max Activating Token Index: 658

Text #16

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5664. Min Act: -0.1700

Data Index: 7752398 (Open Web Text)

Max Activating Token Index: 276

Text #17

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5468. Min Act: -0.1700

Data Index: 473391 (Open Web Text)

Max Activating Token Index: 425

Text #18

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5125. Min Act: -0.1700

Data Index: 4153190 (Open Web Text)

Max Activating Token Index: 551

Text #19

Max Range: 3.7053. Min Range: -3.7053

Max Act: 3.5602. Min Act: -0.1700

Data Index: 2999657 (Open Web Text)

Max Activating Token Index: 588