Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2867 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.9066. Min Act: -0.0000

Data Index: 1006687 (The Pile)

Max Activating Token Index: 241

Text #1

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8265. Min Act: -0.0000

Data Index: 320617 (The Pile)

Max Activating Token Index: 611

Text #2

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8299. Min Act: -0.0000

Data Index: 1903831 (The Pile)

Max Activating Token Index: 284

Text #3

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8373. Min Act: -0.0000

Data Index: 340713 (The Pile)

Max Activating Token Index: 318

Text #4

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8172. Min Act: -0.0000

Data Index: 204425 (The Pile)

Max Activating Token Index: 915

Text #5

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7803. Min Act: -0.0000

Data Index: 1107661 (The Pile)

Max Activating Token Index: 391

Text #6

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8055. Min Act: -0.0000

Data Index: 1843066 (The Pile)

Max Activating Token Index: 589

Text #7

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7985. Min Act: -0.0000

Data Index: 1326997 (The Pile)

Max Activating Token Index: 890

Text #8

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.8072. Min Act: -0.0000

Data Index: 1957957 (The Pile)

Max Activating Token Index: 609

Text #9

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7287. Min Act: -0.0000

Data Index: 699448 (The Pile)

Max Activating Token Index: 968

Text #10

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7454. Min Act: -0.0000

Data Index: 1324779 (The Pile)

Max Activating Token Index: 527

Text #11

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.6790. Min Act: -0.0000

Data Index: 957498 (The Pile)

Max Activating Token Index: 413

Text #12

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7409. Min Act: -0.0000

Data Index: 355382 (The Pile)

Max Activating Token Index: 181

Text #13

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7217. Min Act: -0.0000

Data Index: 1152838 (The Pile)

Max Activating Token Index: 791

Text #14

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.6370. Min Act: -0.0000

Data Index: 1543056 (The Pile)

Max Activating Token Index: 827

Text #15

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7000. Min Act: -0.0000

Data Index: 1796577 (The Pile)

Max Activating Token Index: 64

Text #16

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7385. Min Act: -0.0000

Data Index: 952345 (The Pile)

Max Activating Token Index: 611

Text #17

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.6657. Min Act: -0.0000

Data Index: 1593131 (The Pile)

Max Activating Token Index: 331

Text #18

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.6856. Min Act: -0.0000

Data Index: 243474 (The Pile)

Max Activating Token Index: 711

Text #19

Max Range: 0.9066. Min Range: -0.9066

Max Act: 0.7000. Min Act: -0.0000

Data Index: 954585 (The Pile)

Max Activating Token Index: 407