YuOpera-Omni for multimodal retrieval and constraint aware generation in Henan Opera

Henan Yu Opera is a highly stylized form of intangible cultural heritage whose performance knowledge is distributed across gesture, vocal delivery, rhythmic organization, and lyric form. Existing general multimodal and language models lack domain grounding and often fail to satisfy opera-specific structural constraints, limiting their value for archive analysis, pedagogy, and content creation. We propose YuOpera-Omni, a heritage-oriented multimodal framework that links fine-grained perception, structured-attribute retrieval, and constraint-aware generation within a unified pipeline. The system extracts interpretable opera attributes from video, audio, and text, leverages them to retrieve technically compatible references from a multimodal knowledge base, and generates evidence-grounded critiques and rhyme-compliant lyric drafts under explicit formal constraints. We also construct a domain-oriented benchmark with aligned recordings, lyrics, metadata, and expert annotations. Experiments show that YuOpera-Omni improves retrieval relevance, reduces hallucination, and achieves high rhyme accuracy, supporting digitally grounded Yu Opera understanding and preservation.