AI SAFETY & ETHICS
The Refined Counterfactual Prisoner's Dilemma: An Attempt to Explode Decision-Theoretic Consequentialism
Alignment Forum
•
I was inspired to revise my formulation of this thought experiment by Ihor Kendiukho's post On The Independence Axiom. Kendiukho quotes Scott Garrabrant: My take is that the concept of expected utility maximization is a mistake. As far as I know, every argument for utility assumes (or implies) that whenever you make an observation, you stop caring about the possible worlds where that observation went differently. Von Neumann did not notice this mistake because he was too busy inventing the entire field.