Training diffusion models with low forward-marginal error doesn't guarantee stable sampling—you need additional safeguards like denoiser projection to ensure numerical stability and convergence of sample moments.
This paper reveals a critical gap in diffusion model training: a score function can have tiny errors on average (as measured during training) yet produce numerically unstable sampling with diverging moments. The authors prove this theoretically and show that projecting learned denoisers onto known data bounds fixes the problem.