Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 44 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.4147. Min Act: -0.1700

Data Index: 2414962 (Open Web Text)

Max Activating Token Index: 836

Text #1

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3432. Min Act: -0.1700

Data Index: 834424 (Open Web Text)

Max Activating Token Index: 887

Text #2

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3859. Min Act: -0.1700

Data Index: 4803601 (Open Web Text)

Max Activating Token Index: 317

Text #3

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3788. Min Act: -0.1700

Data Index: 1255416 (Open Web Text)

Max Activating Token Index: 336

Text #4

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3648. Min Act: -0.1700

Data Index: 4372010 (Open Web Text)

Max Activating Token Index: 472

Text #5

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.4054. Min Act: -0.1700

Data Index: 5077043 (Open Web Text)

Max Activating Token Index: 724

Text #6

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3629. Min Act: -0.1700

Data Index: 1082529 (Open Web Text)

Max Activating Token Index: 464

Text #7

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3483. Min Act: -0.1700

Data Index: 2832091 (Open Web Text)

Max Activating Token Index: 787

Text #8

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3198. Min Act: -0.1700

Data Index: 1013995 (Open Web Text)

Max Activating Token Index: 701

Text #9

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.3121. Min Act: -0.1700

Data Index: 7264349 (Open Web Text)

Max Activating Token Index: 1009

Text #10

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2544. Min Act: -0.1700

Data Index: 3982011 (Open Web Text)

Max Activating Token Index: 638

Text #11

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2821. Min Act: -0.1700

Data Index: 4973286 (Open Web Text)

Max Activating Token Index: 937

Text #12

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2362. Min Act: -0.1700

Data Index: 8288459 (Open Web Text)

Max Activating Token Index: 528

Text #13

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2418. Min Act: -0.1700

Data Index: 6124246 (Open Web Text)

Max Activating Token Index: 741

Text #14

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2385. Min Act: -0.1700

Data Index: 7816916 (Open Web Text)

Max Activating Token Index: 850

Text #15

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2312. Min Act: -0.1700

Data Index: 3441813 (Open Web Text)

Max Activating Token Index: 463

Text #16

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2344. Min Act: -0.1700

Data Index: 1373241 (Open Web Text)

Max Activating Token Index: 797

Text #17

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.2250. Min Act: -0.1700

Data Index: 7056706 (Open Web Text)

Max Activating Token Index: 141

Text #18

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.1965. Min Act: -0.1700

Data Index: 6369706 (Open Web Text)

Max Activating Token Index: 850

Text #19

Max Range: 4.4147. Min Range: -4.4147

Max Act: 4.1769. Min Act: -0.1700

Data Index: 1615175 (Open Web Text)

Max Activating Token Index: 693