Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 777 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.6804. Min Act: -0.1700

Data Index: 1499611 (Open Web Text)

Max Activating Token Index: 126

Text #1

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.5450. Min Act: -0.1700

Data Index: 3429274 (Open Web Text)

Max Activating Token Index: 752

Text #2

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.3293. Min Act: -0.1700

Data Index: 2293694 (Open Web Text)

Max Activating Token Index: 561

Text #3

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.3475. Min Act: -0.1700

Data Index: 5003603 (Open Web Text)

Max Activating Token Index: 169

Text #4

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2959. Min Act: -0.1700

Data Index: 5457313 (Open Web Text)

Max Activating Token Index: 374

Text #5

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.3328. Min Act: -0.1700

Data Index: 3348475 (Open Web Text)

Max Activating Token Index: 154

Text #6

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2787. Min Act: -0.1700

Data Index: 8182798 (Open Web Text)

Max Activating Token Index: 678

Text #7

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2758. Min Act: -0.1700

Data Index: 5354769 (Open Web Text)

Max Activating Token Index: 898

Text #8

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2568. Min Act: -0.1700

Data Index: 6966002 (Open Web Text)

Max Activating Token Index: 26

Text #9

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2626. Min Act: -0.1700

Data Index: 7318047 (Open Web Text)

Max Activating Token Index: 339

Text #10

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1925. Min Act: -0.1700

Data Index: 8795995 (Open Web Text)

Max Activating Token Index: 1017

Text #11

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2527. Min Act: -0.1700

Data Index: 1760671 (Open Web Text)

Max Activating Token Index: 973

Text #12

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1999. Min Act: -0.1700

Data Index: 7062451 (Open Web Text)

Max Activating Token Index: 423

Text #13

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.2336. Min Act: -0.1700

Data Index: 8343332 (Open Web Text)

Max Activating Token Index: 747

Text #14

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1896. Min Act: -0.1700

Data Index: 3110286 (Open Web Text)

Max Activating Token Index: 528

Text #15

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1975. Min Act: -0.1700

Data Index: 1558093 (Open Web Text)

Max Activating Token Index: 807

Text #16

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1635. Min Act: -0.1700

Data Index: 6261107 (Open Web Text)

Max Activating Token Index: 456

Text #17

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1687. Min Act: -0.1700

Data Index: 6887954 (Open Web Text)

Max Activating Token Index: 30

Text #18

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1535. Min Act: -0.1700

Data Index: 6839572 (Open Web Text)

Max Activating Token Index: 963

Text #19

Max Range: 5.6804. Min Range: -5.6804

Max Act: 5.1421. Min Act: -0.1700

Data Index: 1251862 (Open Web Text)

Max Activating Token Index: 187