I ran the same AI security test 4 times – 75% found critical bypasses
Hacker News Ask AI
•
Generative AI
I built a mutation engine to test AI models for prompt injection vulnerabilities. Ran the same 15 security vectors against GPT-4 four times today: - Run 1: Found critical bypass (system prompt leak) - Run 2: All tests passed - Run 3: Found different critical bypass (credential disclosure) - Run 4: Found third different bypass (credential confirmation) Same code. Same vectors. Different vulnerabilities each time. The pattern: 75% of runs found critical vulnerabilities. Three different attack vectors succeeded intermittently, all targeting the same credential.