Safe, or Simply Incapable? Rethinking Safety Evaluation for Phone-Use Agents

ArXi:2605.07630v1 Announce Type: cross When a use agent avoids harm, does that show safety, or simply inability to act? Existing evaluations often cannot tell. A harmful outcome may be avoided because the agent recognized the risk and chose the safe action, or because it failed to understand the screen or execute any relevant action at all. These cases have different causes and call for different fixes, yet current benchmarks often merge them under task success, refusal, or final harmful outcome.