Enhancing handwritten mathematical expression recognition with convolutional block attention refinement module and multi-task learning

Handwritten Mathematical Expression Recognition (HMER) is challenging due to complex structural relationships, implicit symbols, and variations in handwriting styles. Therefore, this paper presents a novel approach combining multi-task learning (MTL) with advanced attention mechanisms, including the Convolutional Block Attention Module (CBAM) integrated into the Attention Refinement Module (ARM), to optimize attention mechanisms, enable context-aware interpretation, and improve the generalization of the architecture. Integrating MTL enhances the model's ability to work well on the main task. By sharing knowledge across tasks, the model reduces overfitting and leads to better generalization in the inference stage. Advanced attention mechanisms refine the attention weights to effectively capture highly complex structural expressions and implicit corrections. Our approach addresses the limitations of traditional methods and improves the Expression Recognition Rate on CROHME datasets. Experimental results show that our model has competitive performance and fewer parameters when compared with the state-of-the-art.