The Moral Ceiling of Reinforcement Learning

Psychology classified reward-and-punishment reasoning as the most primitive form of moral development seventy years ago. It remains the most sophisticated behavioural framework in AI. The next breakthrough may come from formalising what lies beyond it. Image by nanobana pro 2 A few years ago I watched a video that changed how I think about artificial intelligence. It was an Atari boat-racing game where the AI agent had been given a single objective: maximise your score. And it did it… a little too well.