PDMP: Rethinking Balanced Multimodal Learning via Performance-Dominant Modality Prioritization

ArXi:2604.05773v1 Announce Type: new Multimodal learning has attracted increasing attention due to its practicality. However, it often suffers from insufficient optimization, where the multimodal model underperforms even compared to its unimodal counterparts. Existing methods attribute this problem to the imbalanced learning between modalities and solve it by gradient modulation. This paper argues that balanced learning is not the optimal setting for multimodal learning.