ABSTRACT Recent breakthroughs in generative AI have markedly elevated the realism and controllability of synthetic media. In the visual modality, long‐context attention mechanisms and diffusion‐style refinements now deliver videos with superior temporal consistency, spatial coherence, and high‐resolution detail. These techniques underpin an expanding set of applications ranging from text‐guided storyboarding and animation to engineering visualization and virtual prototyping. In the audio modality, token‐based representations combined with hierarchical decoding enable the direct production of faithful speech, music, and ambient sound from textual prompts, powering rapid voice‐over creation, personalized music, and immersive soundscapes. The frontier is shifting toward unified audio–visual pipelines that synchronize imagery with dialog, sound effects, and ambience, promising end‐to‐end tooling for a wide variety of applications such as education, simulation, entertainment, and accessible content production. This review surveys these advances across modalities and outlines future research directions focused on improving generation efficiency, coherence, and controllability across modalities.
Charles Ding (2025) studied this question.