Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2554 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2914. Min Act: -0.1700

Data Index: 6289552 (Open Web Text)

Max Activating Token Index: 745

Text #1

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2853. Min Act: -0.1700

Data Index: 274526 (Open Web Text)

Max Activating Token Index: 821

Text #2

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2386. Min Act: -0.1700

Data Index: 5503069 (Open Web Text)

Max Activating Token Index: 449

Text #3

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2390. Min Act: -0.1700

Data Index: 1131366 (Open Web Text)

Max Activating Token Index: 347

Text #4

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2580. Min Act: -0.1700

Data Index: 5640718 (Open Web Text)

Max Activating Token Index: 537

Text #5

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1933. Min Act: -0.1700

Data Index: 4550731 (Open Web Text)

Max Activating Token Index: 543

Text #6

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.2196. Min Act: -0.1700

Data Index: 7541080 (Open Web Text)

Max Activating Token Index: 861

Text #7

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1871. Min Act: -0.1700

Data Index: 6486643 (Open Web Text)

Max Activating Token Index: 768

Text #8

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1578. Min Act: -0.1700

Data Index: 1642806 (Open Web Text)

Max Activating Token Index: 540

Text #9

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1585. Min Act: -0.1700

Data Index: 8403156 (Open Web Text)

Max Activating Token Index: 755

Text #10

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1606. Min Act: -0.1700

Data Index: 8400404 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1730. Min Act: -0.1700

Data Index: 6883449 (Open Web Text)

Max Activating Token Index: 868

Text #12

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1638. Min Act: -0.1700

Data Index: 4734995 (Open Web Text)

Max Activating Token Index: 684

Text #13

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1434. Min Act: -0.1700

Data Index: 2135114 (Open Web Text)

Max Activating Token Index: 1006

Text #14

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1638. Min Act: -0.1700

Data Index: 3458889 (Open Web Text)

Max Activating Token Index: 258

Text #15

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1643. Min Act: -0.1700

Data Index: 8653702 (Open Web Text)

Max Activating Token Index: 996

Text #16

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1510. Min Act: -0.1700

Data Index: 4140511 (Open Web Text)

Max Activating Token Index: 933

Text #17

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1527. Min Act: -0.1700

Data Index: 6836776 (Open Web Text)

Max Activating Token Index: 914

Text #18

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1512. Min Act: -0.1700

Data Index: 8739532 (Open Web Text)

Max Activating Token Index: 672

Text #19

Max Range: 3.2914. Min Range: -3.2914

Max Act: 3.1086. Min Act: -0.1700

Data Index: 6697530 (Open Web Text)

Max Activating Token Index: 549