AI RESEARCH
Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift
arXiv CS.CV
•
ArXi:2603.17372v1 Announce Type: new Large vision-language models (VLMs) often exhibit weakened safety alignment with the integration of the visual modality. Even when text prompts contain explicit harmful intent, adding an image can substantially increase jailbreak success rates. In this paper, we observe that VLMs can clearly distinguish benign inputs from harmful ones in their representation space. Moreover, even among harmful inputs, jailbreak samples form a distinct internal state that is separable from refusal samples.