AI RESEARCH
What Makes VLMs Robust? Towards Reconciling Robustness and Accuracy in Vision-Language Models
arXiv CS.CV
•
ArXi:2603.12799v1 Announce Type: new Achieving adversarial robustness in Vision-Language Models (VLMs) inevitably compromises accuracy on clean data, presenting a long-standing and challenging trade-off. In this work, we revisit this trade-off by investigating a fundamental question: What makes VLMs robust? Through a detailed analysis of adversarially fine-tuned models, we examine how robustness mechanisms function internally and how they interact with clean accuracy. Our analysis reveals that adversarial robustness is not uniformly distributed across network depth.