Multi-Modality Distillation via Learning the teacher's modality-level Gram Matrix

ArXi:2112.11447v2 Announce Type: replace-cross In the context of multi-modality knowledge distillation research, the existing methods was mainly focus on the problem of only learning teacher final output. Thus, there are still deep differences between the teacher network and the student network. It is necessary to force the student network to learn the modality relationship information of the teacher network.