AI RESEARCH
TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition
arXiv CS.LG
•
ArXi:2603.12465v1 Announce Type: cross Large Language Model (LLM) inference is widely used in interactive assistants and agentic systems. In latency-sensitive deployments, inference time can become dominated by host-side overheads. Existing approaches typically expose this cost only as an aggregate residual or a launch/queue metric, which is often insufficient to identify which execution layer should be optimized.