Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 474 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.8760. Min Act: -0.1700

Data Index: 3679601 (Open Web Text)

Max Activating Token Index: 271

Text #1

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7887. Min Act: -0.1700

Data Index: 127690 (Open Web Text)

Max Activating Token Index: 642

Text #2

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7824. Min Act: -0.1700

Data Index: 5586383 (Open Web Text)

Max Activating Token Index: 965

Text #3

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7273. Min Act: -0.1700

Data Index: 6638500 (Open Web Text)

Max Activating Token Index: 515

Text #4

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7268. Min Act: -0.1700

Data Index: 8063854 (Open Web Text)

Max Activating Token Index: 456

Text #5

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7340. Min Act: -0.1700

Data Index: 7207189 (Open Web Text)

Max Activating Token Index: 299

Text #6

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7164. Min Act: -0.1700

Data Index: 626517 (Open Web Text)

Max Activating Token Index: 707

Text #7

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6961. Min Act: -0.1700

Data Index: 1563760 (Open Web Text)

Max Activating Token Index: 455

Text #8

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.7016. Min Act: -0.1700

Data Index: 8036354 (Open Web Text)

Max Activating Token Index: 557

Text #9

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6915. Min Act: -0.1700

Data Index: 3429244 (Open Web Text)

Max Activating Token Index: 695

Text #10

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6899. Min Act: -0.1700

Data Index: 8680704 (Open Web Text)

Max Activating Token Index: 386

Text #11

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6892. Min Act: -0.1700

Data Index: 1729104 (Open Web Text)

Max Activating Token Index: 301

Text #12

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6796. Min Act: -0.1700

Data Index: 5938508 (Open Web Text)

Max Activating Token Index: 325

Text #13

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6790. Min Act: -0.1700

Data Index: 1061050 (Open Web Text)

Max Activating Token Index: 93

Text #14

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6673. Min Act: -0.1700

Data Index: 6775563 (Open Web Text)

Max Activating Token Index: 539

Text #15

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6772. Min Act: -0.1700

Data Index: 6687588 (Open Web Text)

Max Activating Token Index: 854

Text #16

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6617. Min Act: -0.1700

Data Index: 7370194 (Open Web Text)

Max Activating Token Index: 660

Text #17

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6611. Min Act: -0.1700

Data Index: 6114314 (Open Web Text)

Max Activating Token Index: 215

Text #18

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6629. Min Act: -0.1700

Data Index: 2436480 (Open Web Text)

Max Activating Token Index: 718

Text #19

Max Range: 2.8760. Min Range: -2.8760

Max Act: 2.6529. Min Act: -0.1700

Data Index: 2742968 (Open Web Text)

Max Activating Token Index: 911