AI RESEARCH
MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification
arXiv CS.LG
•
ArXi:2601.15498v2 Announce Type: replace Speculative Decoding (SD) accelerates autoregressive large language model (LLM) inference by decoupling generation and verification. While recent methods improve draft quality by tightly coupling the drafter with the target model, the verification mechanism itself remains largely unchanged, relying on strict token-level rejection sampling. In practice, modern LLMs frequently operate in low-margin regimes where the target model exhibits weak preference among top candidates.