AI RESEARCH
Duration Aware Scheduling for ASR Serving Under Workload Drift
arXiv CS.LG
•
ArXi:2603.11273v1 Announce Type: new Scheduling policies in large-scale Automatic Speech Recognition (ASR) serving pipelines play a key role in determining end-to-end (E2E) latency. Yet, widely used serving engines rely on first-come-first-served (FCFS) scheduling, which ignores variability in request duration and leads to head-of-line blocking under workload drift. We show that audio duration is an accurate proxy for job processing time in ASR models such as Whisper, and use this insight to enable duration-aware scheduling.