Uncertainty-Aware Knowledge Distillation for Multimodal Large Language Models

ArXi:2603.21426v1 Announce Type: new Knowledge distillation establishes a learning paradigm that leverages both data supervision and teacher guidance. However, determining the optimal balance between learning from data and learning from the teacher is challenging, as some samples may be noisy while others are subject to teacher uncertainty. This motivates the need for adaptively balancing data and teacher supervision.