Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1477 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0826. Min Act: -0.1700

Data Index: 1606635 (Open Web Text)

Max Activating Token Index: 542

Text #1

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0547. Min Act: -0.1700

Data Index: 3377964 (Open Web Text)

Max Activating Token Index: 998

Text #2

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0399. Min Act: -0.1700

Data Index: 8765349 (Open Web Text)

Max Activating Token Index: 969

Text #3

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0337. Min Act: -0.1700

Data Index: 3377973 (Open Web Text)

Max Activating Token Index: 671

Text #4

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0331. Min Act: -0.1700

Data Index: 8549137 (Open Web Text)

Max Activating Token Index: 588

Text #5

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0135. Min Act: -0.1700

Data Index: 3010692 (Open Web Text)

Max Activating Token Index: 459

Text #6

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0298. Min Act: -0.1700

Data Index: 2746206 (Open Web Text)

Max Activating Token Index: 198

Text #7

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0191. Min Act: -0.1700

Data Index: 1589471 (Open Web Text)

Max Activating Token Index: 660

Text #8

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0554. Min Act: -0.1700

Data Index: 2746209 (Open Web Text)

Max Activating Token Index: 858

Text #9

Max Range: 3.0826. Min Range: -3.0826

Max Act: 2.9691. Min Act: -0.1700

Data Index: 8070623 (Open Web Text)

Max Activating Token Index: 414

Text #10

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0084. Min Act: -0.1700

Data Index: 8484868 (Open Web Text)

Max Activating Token Index: 867

Text #11

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0044. Min Act: -0.1700

Data Index: 1998151 (Open Web Text)

Max Activating Token Index: 698

Text #12

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0034. Min Act: -0.1700

Data Index: 3378107 (Open Web Text)

Max Activating Token Index: 874

Text #13

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0093. Min Act: -0.1700

Data Index: 8765333 (Open Web Text)

Max Activating Token Index: 729

Text #14

Max Range: 3.0826. Min Range: -3.0826

Max Act: 2.9954. Min Act: -0.1700

Data Index: 3010693 (Open Web Text)

Max Activating Token Index: 744

Text #15

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0185. Min Act: -0.1700

Data Index: 1589488 (Open Web Text)

Max Activating Token Index: 663

Text #16

Max Range: 3.0826. Min Range: -3.0826

Max Act: 2.9873. Min Act: -0.1700

Data Index: 1998152 (Open Web Text)

Max Activating Token Index: 203

Text #17

Max Range: 3.0826. Min Range: -3.0826

Max Act: 3.0099. Min Act: -0.1700

Data Index: 3560447 (Open Web Text)

Max Activating Token Index: 449

Text #18

Max Range: 3.0826. Min Range: -3.0826

Max Act: 2.9655. Min Act: -0.1700

Data Index: 3108589 (Open Web Text)

Max Activating Token Index: 458

Text #19

Max Range: 3.0826. Min Range: -3.0826

Max Act: 2.9929. Min Act: -0.1700

Data Index: 7087534 (Open Web Text)

Max Activating Token Index: 178