AI RESEARCH
Scalable AI Inference: Performance Analysis and Optimization of AI Model Serving
arXiv CS.LG
•
ArXi:2604.20420v1 Announce Type: new AI research often emphasizes model design and algorithmic performance, while deployment and inference remain comparatively underexplored despite being critical for real-world use. This study addresses that gap by investigating the performance and optimization of a BentoML-based AI inference system for scalable model serving developed in collaboration with graphworks.ai. The evaluation first establishes baseline performance under three realistic workload scenarios.