Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 25 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.1745. Min Act: -0.1700

Data Index: 4195356 (Open Web Text)

Max Activating Token Index: 922

Text #1

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.1017. Min Act: -0.1700

Data Index: 3423022 (Open Web Text)

Max Activating Token Index: 846

Text #2

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.1224. Min Act: -0.1700

Data Index: 6343970 (Open Web Text)

Max Activating Token Index: 792

Text #3

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0659. Min Act: -0.1700

Data Index: 5682134 (Open Web Text)

Max Activating Token Index: 351

Text #4

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0636. Min Act: -0.1700

Data Index: 6362803 (Open Web Text)

Max Activating Token Index: 217

Text #5

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0579. Min Act: -0.1700

Data Index: 3360905 (Open Web Text)

Max Activating Token Index: 17

Text #6

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0483. Min Act: -0.1700

Data Index: 8404974 (Open Web Text)

Max Activating Token Index: 479

Text #7

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0309. Min Act: -0.1700

Data Index: 4425376 (Open Web Text)

Max Activating Token Index: 65

Text #8

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0202. Min Act: -0.1700

Data Index: 994094 (Open Web Text)

Max Activating Token Index: 311

Text #9

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0344. Min Act: -0.1700

Data Index: 7071087 (Open Web Text)

Max Activating Token Index: 562

Text #10

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0121. Min Act: -0.1700

Data Index: 7171552 (Open Web Text)

Max Activating Token Index: 896

Text #11

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0367. Min Act: -0.1700

Data Index: 1885418 (Open Web Text)

Max Activating Token Index: 517

Text #12

Max Range: 2.1745. Min Range: -2.1745

Max Act: 1.9928. Min Act: -0.1700

Data Index: 8015698 (Open Web Text)

Max Activating Token Index: 351

Text #13

Max Range: 2.1745. Min Range: -2.1745

Max Act: 1.9930. Min Act: -0.1700

Data Index: 2108828 (Open Web Text)

Max Activating Token Index: 736

Text #14

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0102. Min Act: -0.1700

Data Index: 4439125 (Open Web Text)

Max Activating Token Index: 211

Text #15

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0384. Min Act: -0.1700

Data Index: 4115158 (Open Web Text)

Max Activating Token Index: 197

Text #16

Max Range: 2.1745. Min Range: -2.1745

Max Act: 1.9669. Min Act: -0.1700

Data Index: 6165445 (Open Web Text)

Max Activating Token Index: 570

Text #17

Max Range: 2.1745. Min Range: -2.1745

Max Act: 1.9866. Min Act: -0.1700

Data Index: 733977 (Open Web Text)

Max Activating Token Index: 688

Text #18

Max Range: 2.1745. Min Range: -2.1745

Max Act: 2.0210. Min Act: -0.1700

Data Index: 5171032 (Open Web Text)

Max Activating Token Index: 656

Text #19

Max Range: 2.1745. Min Range: -2.1745

Max Act: 1.9938. Min Act: -0.1700

Data Index: 406204 (Open Web Text)

Max Activating Token Index: 963