Product Alignment is not Superintelligence Alignment (and we need the latter to survive)

Tl;dr: progress on making Claude friendly is not the same as progress on making it safe to build godlike superintelligence. solving the former does not imply we get a good future. please track the difference. The term 'Alignment' was coined to point to the technical problem of understanding how to build minds such that if they were to become strongly and generally superhuman, things would go well.