AI RESEARCH
Rotary Masked Autoencoders are Versatile Learners
arXiv CS.LG
•
ArXi:2505.20535v3 Announce Type: replace Applying Transformers to irregular time-series typically requires specializations to their baseline architecture, which can result in additional computational overhead and increased method complexity. We present the Rotary Masked Autoencoder (RoMAE), which utilizes the popular Rotary Positional Embedding (RoPE) method for continuous positions.