Modular Delta Merging with Orthogonal Constraints: A Scalable Framework for Continual and Reversible Model Composition

ArXi:2507.20997v4 Announce Type: replace-cross In real-world machine learning deployments, models must be continually updated, composed, and when required, selectively undone. However, existing approaches to model merging and continual learning often suffer from task interference, catastrophic forgetting, or lack of reversibility. We propose Modular Delta Merging with Orthogonal Constraints (MDM-OC), a novel framework that enables scalable, interference-free, and reversible composition of fine-tuned models.