Do We Really Need Permutations? Impact of Model Width on Linear Mode Connectivity

ArXi:2510.08023v3 Announce Type: replace Recently, Ainsworth empirically nstrated that, given two independently trained models, applying a parameter permutation that preserves the input-output behavior allows the two models to be connected by a low-loss linear path. When such a path exists, the models are said to achieve linear mode connectivity (LMC). Prior studies, including Ainsworth, have reported that achieving LMC requires not only an appropriate permutation search but also sufficiently wide models (e.g., a 32 $\times$ width multiplier for ResNet-20.