AI RESEARCH
Strait: Perceiving Priority and Interference in ML Inference Serving
arXiv CS.LG
•
ArXi:2604.28175v1 Announce Type: new Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. However, limited for task prioritization and insufficient latency estimation under concurrent execution may restrict their applicability in on-premises scenarios. We present \emph{Strait}, a serving system designed to enhance deadline satisfaction for dual-priority inference traffic under high GPU utilization.