AI RESEARCH
Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score
arXiv CS.LG
•
ArXi:2605.02206v1 Announce Type: cross Machine unlearning in Vision-Language Models (VLMs) is required for compliance with the General Data Protection Regulation (GDPR), yet current evaluation practices are inconsistent. We present the first systematic study of metric reliability in multimodal unlearning. Five standard metrics, Forget Accuracy (FA), Retain Accuracy (RA), Membership Inference Attack (MIA), Activation Distance (AD), and JS divergence (JS), yield conflicting method rankings across three VQA benchmarks (MLLMU-Bench, UnLOK-VQA, MMUBench.