AI RESEARCH
RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodings
arXiv CS.LG
•
ArXi:2605.10706v1 Announce Type: new We present a new class of efficient attention mechanisms applying universal 3D Relative Positional Encoding (RPE) methods given by arbitrary integrable modulation functions $f$. They lead to the new class of 3D-Transformer models, called \textit{RelFlexformers}, flexibly integrating those RPEs, and characterized by the $O(L \log L)$ time complexity of the attention computation for the $L$-length input sequences.