Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 22 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.9109. Min Act: -0.1700

Data Index: 7980099 (Open Web Text)

Max Activating Token Index: 13

Text #1

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.9131. Min Act: -0.1700

Data Index: 5819689 (Open Web Text)

Max Activating Token Index: 381

Text #2

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8643. Min Act: -0.1700

Data Index: 5561063 (Open Web Text)

Max Activating Token Index: 581

Text #3

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8572. Min Act: -0.1700

Data Index: 188711 (Open Web Text)

Max Activating Token Index: 632

Text #4

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8457. Min Act: -0.1700

Data Index: 1569905 (Open Web Text)

Max Activating Token Index: 34

Text #5

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8465. Min Act: -0.1700

Data Index: 6062369 (Open Web Text)

Max Activating Token Index: 827

Text #6

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8394. Min Act: -0.1700

Data Index: 2881593 (Open Web Text)

Max Activating Token Index: 568

Text #7

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8233. Min Act: -0.1700

Data Index: 2000311 (Open Web Text)

Max Activating Token Index: 577

Text #8

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8215. Min Act: -0.1700

Data Index: 6065637 (Open Web Text)

Max Activating Token Index: 685

Text #9

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8108. Min Act: -0.1700

Data Index: 4546148 (Open Web Text)

Max Activating Token Index: 581

Text #10

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8348. Min Act: -0.1700

Data Index: 6683128 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.8006. Min Act: -0.1700

Data Index: 3108324 (Open Web Text)

Max Activating Token Index: 17

Text #12

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7929. Min Act: -0.1700

Data Index: 3495770 (Open Web Text)

Max Activating Token Index: 629

Text #13

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7883. Min Act: -0.1700

Data Index: 4598686 (Open Web Text)

Max Activating Token Index: 24

Text #14

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7825. Min Act: -0.1698

Data Index: 2776396 (Open Web Text)

Max Activating Token Index: 376

Text #15

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7844. Min Act: -0.1700

Data Index: 3015008 (Open Web Text)

Max Activating Token Index: 468

Text #16

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7851. Min Act: -0.1700

Data Index: 6362027 (Open Web Text)

Max Activating Token Index: 755

Text #17

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7899. Min Act: -0.1694

Data Index: 5975607 (Open Web Text)

Max Activating Token Index: 590

Text #18

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7689. Min Act: -0.1700

Data Index: 2283572 (Open Web Text)

Max Activating Token Index: 660

Text #19

Max Range: 1.9131. Min Range: -1.9131

Max Act: 1.7674. Min Act: -0.1700

Data Index: 6800006 (Open Web Text)

Max Activating Token Index: 95