Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 79 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.9134. Min Act: -0.1700

Data Index: 8208517 (Open Web Text)

Max Activating Token Index: 468

Text #1

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8627. Min Act: -0.1700

Data Index: 7279809 (Open Web Text)

Max Activating Token Index: 915

Text #2

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8395. Min Act: -0.1700

Data Index: 283794 (Open Web Text)

Max Activating Token Index: 1012

Text #3

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8420. Min Act: -0.1700

Data Index: 2923005 (Open Web Text)

Max Activating Token Index: 834

Text #4

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8220. Min Act: -0.1700

Data Index: 7502259 (Open Web Text)

Max Activating Token Index: 1001

Text #5

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8288. Min Act: -0.1700

Data Index: 1608920 (Open Web Text)

Max Activating Token Index: 958

Text #6

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7992. Min Act: -0.1700

Data Index: 7776989 (Open Web Text)

Max Activating Token Index: 428

Text #7

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.8131. Min Act: -0.1700

Data Index: 2759176 (Open Web Text)

Max Activating Token Index: 470

Text #8

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7979. Min Act: -0.1700

Data Index: 7794664 (Open Web Text)

Max Activating Token Index: 341

Text #9

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7876. Min Act: -0.1700

Data Index: 3733510 (Open Web Text)

Max Activating Token Index: 939

Text #10

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7867. Min Act: -0.1700

Data Index: 2062026 (Open Web Text)

Max Activating Token Index: 864

Text #11

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7919. Min Act: -0.1700

Data Index: 3127130 (Open Web Text)

Max Activating Token Index: 517

Text #12

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7817. Min Act: -0.1700

Data Index: 8472779 (Open Web Text)

Max Activating Token Index: 117

Text #13

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7803. Min Act: -0.1700

Data Index: 5274408 (Open Web Text)

Max Activating Token Index: 552

Text #14

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7677. Min Act: -0.1700

Data Index: 2154365 (Open Web Text)

Max Activating Token Index: 1008

Text #15

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7730. Min Act: -0.1700

Data Index: 6986711 (Open Web Text)

Max Activating Token Index: 995

Text #16

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7621. Min Act: -0.1700

Data Index: 6667933 (Open Web Text)

Max Activating Token Index: 448

Text #17

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7520. Min Act: -0.1700

Data Index: 3707172 (Open Web Text)

Max Activating Token Index: 783

Text #18

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7439. Min Act: -0.1700

Data Index: 7004729 (Open Web Text)

Max Activating Token Index: 965

Text #19

Max Range: 1.9134. Min Range: -1.9134

Max Act: 1.7525. Min Act: -0.1700

Data Index: 5427889 (Open Web Text)

Max Activating Token Index: 252