Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1513 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.3016. Min Act: -0.1700

Data Index: 7662457 (Open Web Text)

Max Activating Token Index: 736

Text #1

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.3379. Min Act: -0.1700

Data Index: 1500540 (Open Web Text)

Max Activating Token Index: 524

Text #2

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.3247. Min Act: -0.1700

Data Index: 735581 (Open Web Text)

Max Activating Token Index: 158

Text #3

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.2878. Min Act: -0.1700

Data Index: 8317179 (Open Web Text)

Max Activating Token Index: 923

Text #4

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.2477. Min Act: -0.1700

Data Index: 4214043 (Open Web Text)

Max Activating Token Index: 546

Text #5

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.2093. Min Act: -0.1700

Data Index: 3716517 (Open Web Text)

Max Activating Token Index: 170

Text #6

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.2162. Min Act: -0.1700

Data Index: 731299 (Open Web Text)

Max Activating Token Index: 365

Text #7

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1861. Min Act: -0.1700

Data Index: 3521448 (Open Web Text)

Max Activating Token Index: 643

Text #8

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1955. Min Act: -0.1700

Data Index: 4095065 (Open Web Text)

Max Activating Token Index: 519

Text #9

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1849. Min Act: -0.1700

Data Index: 1122712 (Open Web Text)

Max Activating Token Index: 557

Text #10

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.2021. Min Act: -0.1700

Data Index: 1414787 (Open Web Text)

Max Activating Token Index: 544

Text #11

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1669. Min Act: -0.1700

Data Index: 1173262 (Open Web Text)

Max Activating Token Index: 540

Text #12

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1430. Min Act: -0.1700

Data Index: 4105611 (Open Web Text)

Max Activating Token Index: 333

Text #13

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1684. Min Act: -0.1700

Data Index: 3869413 (Open Web Text)

Max Activating Token Index: 541

Text #14

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1623. Min Act: -0.1700

Data Index: 7133428 (Open Web Text)

Max Activating Token Index: 451

Text #15

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1614. Min Act: -0.1700

Data Index: 6923082 (Open Web Text)

Max Activating Token Index: 375

Text #16

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1744. Min Act: -0.1700

Data Index: 6751278 (Open Web Text)

Max Activating Token Index: 479

Text #17

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1684. Min Act: -0.1700

Data Index: 4269156 (Open Web Text)

Max Activating Token Index: 120

Text #18

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1168. Min Act: -0.1700

Data Index: 1784237 (Open Web Text)

Max Activating Token Index: 431

Text #19

Max Range: 6.3379. Min Range: -6.3379

Max Act: 6.1107. Min Act: -0.1700

Data Index: 1056612 (Open Web Text)

Max Activating Token Index: 401