AI RESEARCH
From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
arXiv CS.CL
•
ArXi:2604.15244v1 Announce Type: new Speculative decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose outputs that a stronger target model verifies. However, its token-centric nature allows erroneous steps to propagate. Prior approaches mitigate this using external reward models, but incur additional latency, computational overhead, and limit generalizability. We propose SpecGuard, a verification-aware speculative decoding framework that performs step-level verification using only model-internal signals.