AI RESEARCH
The Silent Hyperparameter: Quantifying the Impact of Inference Backends on LLM Reproducibility
arXiv CS.LG
•
ArXi:2605.19537v1 Announce Type: new Progress in LLMs is increasingly measured through standardized benchmarks, where state-of-the-art improvements are often separated by fractions of a percentage point. At the same time, the computational cost of evaluating modern LLMs has driven widespread adoption of specialized inference backends, software systems that execute trained models efficiently at inference time. While critical for scalability, system-level optimizations, such as custom CUDA kernels and reduced-precision arithmetic, can alter token probabilities and.