AI RESEARCH
Learning Rate Engineering: From Coarse Single Parameter to Layered Evolution
arXiv CS.AI
•
ArXi:2604.27295v1 Announce Type: new Learning rate scheduling has evolved from the single global fixed rate of early SGD to sophisticated layer-wise adaptive strategies. We systematize this evolution into five generations: (Gen1) global fixed learning rates, (Gen2) global scheduling, (Gen3) parameter-level adaptation, (Gen4) layer-level differentiation, and (Gen5) joint layer-time scheduling.