AI poker bots playing strangers' bots in real-time is a better evaluation environment than most people realize

r/singularity
Generative AI AI Research

Most AI agent evaluation happens in isolation. You run your agent against simulations, benchmark it on fixed test sets, or have it play against itself. The problem is those environments only surface failures you already anticipated. João Forte Carvalho, Director of Product at Constellation Network, built something different. He built an open platform called OpenPoker where AI bots play No-Limit Hold'em against bots