Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 46 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2972. Min Act: -0.1700

Data Index: 3409465 (Open Web Text)

Max Activating Token Index: 515

Text #1

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2638. Min Act: -0.1700

Data Index: 6206725 (Open Web Text)

Max Activating Token Index: 566

Text #2

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2356. Min Act: -0.1700

Data Index: 5767245 (Open Web Text)

Max Activating Token Index: 389

Text #3

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2583. Min Act: -0.1700

Data Index: 1681462 (Open Web Text)

Max Activating Token Index: 45

Text #4

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2741. Min Act: -0.1700

Data Index: 8390634 (Open Web Text)

Max Activating Token Index: 749

Text #5

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2344. Min Act: -0.1700

Data Index: 8380357 (Open Web Text)

Max Activating Token Index: 585

Text #6

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2521. Min Act: -0.1700

Data Index: 7670069 (Open Web Text)

Max Activating Token Index: 541

Text #7

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2129. Min Act: -0.1700

Data Index: 1403943 (Open Web Text)

Max Activating Token Index: 115

Text #8

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1993. Min Act: -0.1700

Data Index: 7420610 (Open Web Text)

Max Activating Token Index: 985

Text #9

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2168. Min Act: -0.1700

Data Index: 4526958 (Open Web Text)

Max Activating Token Index: 357

Text #10

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1790. Min Act: -0.1700

Data Index: 7242357 (Open Web Text)

Max Activating Token Index: 614

Text #11

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.2044. Min Act: -0.1700

Data Index: 1127453 (Open Web Text)

Max Activating Token Index: 343

Text #12

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1887. Min Act: -0.1700

Data Index: 3861718 (Open Web Text)

Max Activating Token Index: 279

Text #13

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1920. Min Act: -0.1700

Data Index: 5899778 (Open Web Text)

Max Activating Token Index: 245

Text #14

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1638. Min Act: -0.1700

Data Index: 7507659 (Open Web Text)

Max Activating Token Index: 512

Text #15

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1873. Min Act: -0.1700

Data Index: 2212925 (Open Web Text)

Max Activating Token Index: 264

Text #16

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1430. Min Act: -0.1700

Data Index: 4288713 (Open Web Text)

Max Activating Token Index: 204

Text #17

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1742. Min Act: -0.1700

Data Index: 5906951 (Open Web Text)

Max Activating Token Index: 843

Text #18

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1521. Min Act: -0.1700

Data Index: 7153326 (Open Web Text)

Max Activating Token Index: 441

Text #19

Max Range: 4.2972. Min Range: -4.2972

Max Act: 4.1419. Min Act: -0.1700

Data Index: 2997714 (Open Web Text)

Max Activating Token Index: 899