Educational PyTorch repo for distributed training from scratch: DP, FSDP, TP, FSDP+TP, and PP
r/artificial
•
AI Tools
I put together a small educational repo that implements distributed