Answers are grounded in this paper and cite their sources.
Unified image understanding and text-conditioned image generation require models to coordinate discriminative semantic representations with generation-compatible latent representations. Existing approaches often rely on either strongly shared representations, which may cause task interference, or separated branches, which may weaken semantic transfer. This paper proposes CSB-UG, a unified understanding–generation framework based on a cross-modal semantic bridge. CSB-UG projects visual features, textual semantics, and generative latents into an intermediate bridge space, while task-adaptive adapters preserve task-specific transformations. It further introduces controlled bidirectional collaboration and a patch-token consistency objective to enhance semantic guidance, latent feedback, and local alignment. Experiments on MS-COCO Captions, VQAv2, Visual Genome, and GenEval evaluate retrieval, visual question answering, captioning, text-to-image generation, generation alignment, and fine-grained semantic consistency. Results show that CSB-UG achieves competitive performance across the evaluated understanding and generation tasks, with improvements in retrieval, captioning, generation alignment, and local semantic consistency. Qualitative analyses further suggest improved constraint preservation and object-, attribute-, and relation-level alignment, while failure cases show that complex spatial relations, counting, long-prompt following, and rare attributes remain challenging. These results indicate that bridge-based semantic coordination is a useful design choice for unified image understanding and generation under the evaluated settings.
No takes yet. Share an insight, caveat, or question.
Xiaoping Gu (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: