AI RESEARCH
ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving
arXiv CS.LG
•
ArXi:2604.00136v1 Announce Type: new Production LLM serving often relies on multi-model portfolios spanning a ~530x cost range, where routing decisions trade off quality against cost. This trade-off is non-stationary: providers revise pricing, model quality can regress silently, and new models must be integrated without downtime. We present ParetoBandit, an open-source adaptive router built on cost-aware contextual bandits that is the first to simultaneously enforce dollar-denominated budgets, adapt online to such shifts, and onboard new models at runtime.