Medical Ontology can significantly enhance the performance of mainstream text embedding models in retrieving medical information: Exploration research (Preprint)

BACKGROUND In recent years, text embedding models and their associated vector search techniques have seen significant advancements. These technologies have become crucial for deploying large language models in specialized fields like medicine. However, the inherent complexity of domain-specific knowledge still poses challenges for mainstream text embedding models. These models often struggle to accurately interpret and represent specialized terminology. The medical field, with its extensive and diverse terminology, highlights a gap in systematically evaluating how this terminological diversity affects the precision of vector-based information retrieval for text embedding models. OBJECTIVE This study seeks to evaluate how the diversity of medical terminology affects vector-based information retrieval for text embedding models. Additionally, it aims to identify and propose potential strategies to mitigate these effects. METHODS We initially developed a medical knowledge base comprising 9715 sentences, utilizing phenotypic terms and their definitions from the Human Phenotype Ontology. This knowledge base was then vectorized using 19 representative text embedding models and stored in a vector database separately. Following this, we conducted a series of information retrieval experiments to assess the efficacy of vector-based retrieval with both preferred and non-preferred phenotypic terms across the different text embedding models. This approach allowed us to thoroughly analyze how the diversity of medical terminology affects retrieval accuracy. RESULTS Our findings reveal that the diversity of medical terminology substantially affects the vector-based information retrieval accuracy of text embedding models for medical texts. Besides, it is found that the greater the literal or semantic differences between preferred and non-preferred terms for a given medical concept, the more significant the impact on vector-based information retrieval accuracy. Furthermore, current text embedding models demonstrate markedly weaker performance in retrieving cross-lingual medical texts compared to texts within the same language. Utilizing medical ontologies to standardize concepts in medical texts can significantly improve the vector-based information retrieval accuracy of these models. CONCLUSIONS The diversity of medical terminology has a profound impact on the vector-based information retrieval accuracy of text embedding models for medical texts. Employing medical ontologies to standardize the concepts within these texts represents a highly effective strategy for enhancing the retrieval accuracy of existing text embedding models.