AI RESEARCH
Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models
arXiv CS.CL
•
ArXi:2505.24187v2 Announce Type: replace The prevailing assumption of an exponential decay in large language model (LLM) reliability with sequence length, predicated on independent per-token error probabilities, posits an inherent limitation for long autoregressive outputs. Our research fundamentally challenges this view by synthesizing emerging evidence that LLM errors are not uniformly distributed but are concentrated at sparse "key tokens" ($5-10\%$ of total tokens) representing critical decision junctions.