Multimodal fusion with supervised sparse autoencoding for high-performance seal retrieval in digital heritage

Accurate seal image retrieval is a foundational task in constructing knowledge graphs for Chinese painting provenance, enabling the reliable establishment of collector–artwork and artist–artwork relationship triples. However, existing methods suffer from insufficient robustness against visual noise and physical wear, and struggle with the semantic ambiguity of leisure seals. This paper proposes a framework combining multimodal fusion embedding representation and supervised sparse dimensionality reduction to address these challenges. First, we integrate a general visual encoder, a symbol feature encoder, and a vision–language model to generate joint embeddings through feature concatenation, significantly enhancing the representation of seal visual structures, symbolic features, and cultural semantics. Second, we design a supervised sparse autoencoder (SSAE) based on contrastive learning, utilising a k -sparsification function and a cosine-similarity-oriented architecture to compress high-dimensional fused features into low-dimensional sparse embeddings. This approach reduces storage overhead by 48% while achieving a 4.3-fold acceleration in retrieval speed compared to the baseline. Experiments on the self-constructed Chinese painting seal dataset SealArt-1K demonstrate that our method achieves 84.6% mAP@1 and 94.7% Recall@10 on the full test set, with 78.3% mAP@1 on severely worn seals, demonstrating strong robustness in real-world digital heritage scenarios. The SSAE dimensionality reduction maintains comparable performance to the full-dimensional baseline while enabling high-throughput deployment, outperforming existing methods on both worn seals and leisure seals with semantic ambiguity.