A survey: advances in multi-modal visual understanding and generation

Abstract Visual generation and understanding have rapidly advanced in recent years, propelled by deep learning and the growing availability of large-scale multimodal datasets. Among the most transformative developments are diffusion models, which have achieved state-of-the-art results in image and video generation through a stochastic denoising process. Beyond high-fidelity synthesis, diffusion models have shown remarkable flexibility for controllable generation, multi-modal conditioning, and even visual understanding tasks such as depth estimation and segmentation. In this survey, we provide a comprehensive overview of recent progress at the intersection of diffusion-based modeling and multi-modal visual intelligence. We structure our discussion around five key themes: (1) diffusion-based image and video generation, (2) controllable diffusion via structured conditions, (3) diffusion for visual understanding, (4) multi-modal generation beyond RGB, and (5) unified frameworks for generation and understanding. By reviewing advances across these domains, we aim to outline emerging trends, identify open challenges, and inspire future research toward more general and unified models of visual perception and synthesis.