Answers are grounded in this paper and cite their sources.
Cataloged artworks combine images, descriptions, and labels, but no single source fully conveys an artwork’s subject and style. An image may leave a narrative subject ambiguous, and a description may omit important visual details. Processing these sources separately misses complementary evidence for classification and retrieval and gives a later generation step no prompts grounded in that evidence. Fixed averaging or concatenation of pretrained features cannot adapt the contribution of each source to the artwork at hand. We propose MF-CAG, a two-stage framework that links multimodal artwork analysis with retrieval-guided image generation. Frozen CLIP and BERT encoders extract visual and textual features, and a trainable style-aware fusion module projects them, together with school-label embeddings, into a shared space and combines them with artwork-specific modality weights. The fused representation supports theme classification and semantic retrieval, and terms from the descriptions of retrieved artworks fill a structured prompt for a frozen Stable Diffusion model. On the SemArt benchmark, MF-CAG reaches an Accuracy of 0.8499 and a Macro-F1 of 0.7930 in theme (TYPE) classification while training only 1.43 M parameters. It also achieves the best retrieval scores among the compared methods and raises the CLIP Score of generated images from 0.2714 with label-only prompts to 0.3521.
No takes yet. Share an insight, caveat, or question.
Wang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: