AI RESEARCH
Towards Shutdownable Agents: Generalizing Stochastic Choice in RL Agents and LLMs
arXiv CS.AI
•
ArXi:2604.17502v1 Announce Type: new Misaligned artificial agents might resist shutdown. One proposed solution is to train agents to lack preferences between different-length trajectories. The Discounted Reward for Same-Length Trajectories (DReST) reward function does this by penalizing agents for repeatedly choosing same-length trajectories, and thus incentivizes agents to (1) choose stochastically between different trajectory-lengths (be Neutral about trajectory-lengths), and (2) pursue goals effectively conditional on each trajectory-length (be Useful.