AI RESEARCH
Learning Variable-Length Tokenization for Generative Recommendation
arXiv CS.LG
•
ArXi:2605.17779v1 Announce Type: new Generative recommendation reformulates recommendation as next-token prediction over discrete semantic identifiers (IDs). A fundamental yet unexplored design choice is that existing methods employ fixed-length tokenization for all items, implicitly assuming uniform encoding capacity regardless of item characteristics.