Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 944 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.1528. Min Act: -0.1700

Data Index: 4740637 (Open Web Text)

Max Activating Token Index: 982

Text #1

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.1400. Min Act: -0.1700

Data Index: 1512089 (Open Web Text)

Max Activating Token Index: 960

Text #2

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.1111. Min Act: -0.1700

Data Index: 6413720 (Open Web Text)

Max Activating Token Index: 874

Text #3

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.0422. Min Act: -0.1700

Data Index: 1358198 (Open Web Text)

Max Activating Token Index: 709

Text #4

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.0387. Min Act: -0.1700

Data Index: 8124984 (Open Web Text)

Max Activating Token Index: 885

Text #5

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9776. Min Act: -0.1700

Data Index: 2112989 (Open Web Text)

Max Activating Token Index: 498

Text #6

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9807. Min Act: -0.1700

Data Index: 7189828 (Open Web Text)

Max Activating Token Index: 473

Text #7

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.0055. Min Act: -0.1700

Data Index: 6149846 (Open Web Text)

Max Activating Token Index: 963

Text #8

Max Range: 4.1528. Min Range: -4.1528

Max Act: 4.0030. Min Act: -0.1700

Data Index: 7390399 (Open Web Text)

Max Activating Token Index: 508

Text #9

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9787. Min Act: -0.1700

Data Index: 6149851 (Open Web Text)

Max Activating Token Index: 480

Text #10

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9502. Min Act: -0.1700

Data Index: 1769583 (Open Web Text)

Max Activating Token Index: 375

Text #11

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9619. Min Act: -0.1700

Data Index: 879742 (Open Web Text)

Max Activating Token Index: 1021

Text #12

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9442. Min Act: -0.1700

Data Index: 4596939 (Open Web Text)

Max Activating Token Index: 430

Text #13

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9316. Min Act: -0.1700

Data Index: 4083947 (Open Web Text)

Max Activating Token Index: 501

Text #14

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9166. Min Act: -0.1700

Data Index: 5642772 (Open Web Text)

Max Activating Token Index: 514

Text #15

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9333. Min Act: -0.1700

Data Index: 8322406 (Open Web Text)

Max Activating Token Index: 149

Text #16

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9260. Min Act: -0.1700

Data Index: 3925545 (Open Web Text)

Max Activating Token Index: 114

Text #17

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9254. Min Act: -0.1700

Data Index: 6479088 (Open Web Text)

Max Activating Token Index: 743

Text #18

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9122. Min Act: -0.1700

Data Index: 1771286 (Open Web Text)

Max Activating Token Index: 999

Text #19

Max Range: 4.1528. Min Range: -4.1528

Max Act: 3.9143. Min Act: -0.1700

Data Index: 742556 (Open Web Text)

Max Activating Token Index: 851