AI RESEARCH
Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inference
arXiv CS.CL
•
ArXi:2605.11581v1 Announce Type: new When large language models (LLMs) serve real-time inference in commercial online advertising systems, end-to-end latency must be strictly bounded to the millisecond range. Yet every token generated during the decode phase triggers thousands of kernel launches, and kernel launch overhead alone can account for 14.6% of end-to-end inference time. MegaKernel eliminates launch overhead and inter-operator HBM round-trips by fusing multiple operators into a single persistent kernel.