AI RESEARCH
Unveiling the Basin-Like Loss Landscape in Large Language Models
arXiv CS.LG
•
ArXi:2505.17646v3 Announce Type: replace We discover the emergence of \textit{basins} in the loss landscape of large language models. As model scale increases, LLMs become progressively resilient to random perturbations in the parameter space, giving rise to expansive stability regions where models exhibit nearly identical performance, but outside of which their capabilities collapse. We observe that pre-