Collaborative Mixture of Experts for Enhanced Code-Switching Visual Recognition Through Recursive Token Propagation

Abstract Multilingual environments often require models to accurately recognize and process both linguistic and visual inputs, particularly in cases where code-switching occurs frequently. Traditional models designed for language and visual recognition struggle to handle such complexities due to their inability to dynamically allocate resources to specific tasks. The introduction of a Collaborative Mixture of Experts (MoE) model within Mistral 8x7b addresses this limitation through the implementation of expert modules that specialize in different aspects of code-switching and visual recognition. A dynamic gating mechanism selects the most appropriate expert based on the input, leading to substantial improvements in precision, recall, and cross-modal accuracy. Experimental results demonstrate that the MoE architecture not only enhances linguistic boundary detection in multilingual contexts but also significantly improves the alignment between language and visual data, ensuring more robust and adaptive performance in real-time applications. The findings indicate that the proposed model outperforms baseline models in both accuracy and computational efficiency, suggesting its potential for further advancements in handling complex multimodal tasks.