The explosive growth of academic literature leads to the problem of semantic gap and multi-modal information fragmentation in traditional retrieval system. In this study, a multi-dimensional semantic indexing and optimization algorithm framework integrating text, formula, chart and citation relationship is proposed. Firstly, the framework uses SPECTER, Formula Encoder, CLIP and GraphSAGE to extract the heterogeneous features of documents, and designs a gated cross-attention network to dynamically fuse the modal information based on the text to generate a unified semantic vector. Secondly, a hierarchical main-incremental double index structure based on HNSW (hierarchical navigable small world) is constructed, which supports millisecond retrieval and real-time update of hundreds of millions of documents, and realizes asynchronous local optimization by combining local sensitive hash. Furthermore, the continuous learning mechanism is introduced, and the dynamic evolution of scientific research frontier terms is tracked by using the Elastic weight consolidation (EWC) algorithm. The experiment is based on S2ORC data set (5 million documents) and complex multimodal query set. The evaluation shows that the proposed framework nDCG@10 reaches 0.418, which is 18.8% higher than the baseline. The query delay of tens of millions of data is less than 20ms, and the incremental update throughput is nearly 1000 articles/second, which effectively realizes cross-modal deep semantic correlation and system dynamic adaptability.
Cong et al. (Sun,) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: