Discrete diffusion models can now be distilled into faster generators using moment matching, enabling practical deployment with fewer sampling steps while maintaining quality.
This paper solves the problem of making discrete diffusion models faster by distilling them into simpler models. Unlike continuous diffusion models which have many distillation techniques, discrete diffusion (used for text and images) has been hard to compress.