Sentence Embedding Using Multimodal Approach: Combining FastText with AraBERT for Arabic Text Representation

Background Sentence-embedding models transform sentences into dense vector representations that capture their semantic meanings. These representations enable deep learning to perform many tasks efficiently, such as similarity measurement, retrieval, and summarization, with improved semantic understanding. Existing sentence embedding models often struggle to capture the semantic richness and morphological complexity of Arabic, limiting their effectiveness in tasks such as semantic similarity, question answering, summarization, and information retrieval. Objectives This study aims to develop a novel sentence-embedding framework tailored for Arabic that addresses the shortcomings of current models by integrating contextual and linguistic features. Methods We propose a multimodal architecture that combines a fine-tuned Sentence-AraBERT (SAraBERT) model with pre-trained FastText embeddings. The model is evaluated on standard Arabic Semantic Textual Similarity (STS) benchmarks using the Mean Squared Error (MSE) and Pearson Correlation Coefficient. Results Experimental results show that the proposed model outperforms existing baselines, achieving lower MSE values (0.0355) and higher correlation scores (0.8053), indicating a stronger alignment with human-annotated similarity judgments on the ATrD dataset. Conclusion The findings demonstrate the effectiveness of multimodal SAraBERT-based embeddings in enhancing sentence-level semantic understanding of Arabic. This study advances Natural Language Processing (NLP) capabilities for underrepresented languages and provides a foundation for future research on Arabic language understanding using deep learning techniques.