AI RESEARCH
Achieving Linear Speedup with ProxSkip in Distributed Stochastic Optimization
arXiv CS.LG
•
ArXi:2310.07983v5 Announce Type: replace The ProxSkip algorithm for distributed optimization is gaining increasing attention due to its effectiveness in reducing communication. However, existing analyses of ProxSkip are limited to the strongly convex setting and fail to achieve linear speedup with respect to the number of nodes. Key questions regarding its behavior in the non-convex setting and the achievability of linear speedup remain open. In this paper, we revisit decentralized ProxSkip and answer these questions affirmatively.