AI RESEARCH
PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models
arXiv CS.AI
•
ArXi:2501.03544v5 Announce Type: replace-cross Recent text-to-image (T2I) models have exhibited remarkable performance in generating high-quality images from text descriptions. However, these models are vulnerable to misuse, particularly generating not-safe-for-work (NSFW) content, such as sexually explicit, violent, political, and disturbing images, raising serious ethical concerns. In this work, we present PromptGuard, a novel content moderation technique that draws inspiration from the system prompt mechanism in large language models (LLMs) for safety alignment.