[New Optimizer] 🌹 Rose: low VRAM, easy to use, great results, Apache 2.0 [P]

Hello, World! I recently released a new PyTorch optimizer I've been researching and developing on my own for the last couple of years. It's named "Rose" in memory of my mother, who loved to hear about my discoveries and progress with AI. Without going too much into the technical details (which you can read about in the GitHub repo), here are some of its benefits: It's stateless, which means it uses less memory than even 8-bit AdamW. If it weren't for temporary working memory, its memory use would be as low as plain vanilla SGD ( without momentum.