AI RESEARCH
Frayed RoPE and Long Inputs: A Geometric Perspective
arXiv CS.LG
•
ArXi:2603.18017v1 Announce Type: new Rotary Positional Embedding (RoPE) is a widely adopted technique for encoding position in language models, which, while effective, causes performance breakdown when input length exceeds