Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1654 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.9527. Min Act: -0.1700

Data Index: 1383643 (Open Web Text)

Max Activating Token Index: 645

Text #1

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8109. Min Act: -0.1700

Data Index: 4092946 (Open Web Text)

Max Activating Token Index: 795

Text #2

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8492. Min Act: -0.1700

Data Index: 676636 (Open Web Text)

Max Activating Token Index: 144

Text #3

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8648. Min Act: -0.1700

Data Index: 6498460 (Open Web Text)

Max Activating Token Index: 8

Text #4

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8634. Min Act: -0.1700

Data Index: 2762081 (Open Web Text)

Max Activating Token Index: 222

Text #5

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7881. Min Act: -0.1700

Data Index: 7947201 (Open Web Text)

Max Activating Token Index: 517

Text #6

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8216. Min Act: -0.1700

Data Index: 5292563 (Open Web Text)

Max Activating Token Index: 431

Text #7

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8246. Min Act: -0.1700

Data Index: 900109 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.8446. Min Act: -0.1700

Data Index: 635617 (Open Web Text)

Max Activating Token Index: 594

Text #9

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7613. Min Act: -0.1700

Data Index: 8087170 (Open Web Text)

Max Activating Token Index: 479

Text #10

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7913. Min Act: -0.1700

Data Index: 6396743 (Open Web Text)

Max Activating Token Index: 710

Text #11

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7657. Min Act: -0.1700

Data Index: 5812635 (Open Web Text)

Max Activating Token Index: 490

Text #12

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7653. Min Act: -0.1700

Data Index: 4936586 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7889. Min Act: -0.1700

Data Index: 4261556 (Open Web Text)

Max Activating Token Index: 854

Text #14

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7653. Min Act: -0.1700

Data Index: 8547005 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7775. Min Act: -0.1700

Data Index: 7063321 (Open Web Text)

Max Activating Token Index: 122

Text #16

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7654. Min Act: -0.1700

Data Index: 3239481 (Open Web Text)

Max Activating Token Index: 644

Text #17

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7446. Min Act: -0.1700

Data Index: 5335718 (Open Web Text)

Max Activating Token Index: 774

Text #18

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7617. Min Act: -0.1700

Data Index: 7113282 (Open Web Text)

Max Activating Token Index: 33

Text #19

Max Range: 4.9527. Min Range: -4.9527

Max Act: 4.7257. Min Act: -0.1700

Data Index: 1933306 (Open Web Text)

Max Activating Token Index: 622