A Retrieval Augmentation Self-Distillation Method for Math Word Problem Solving

Solving math word problems automatically is a critical task in the field of natural language processing. Due to the insufficient size of existing MWP datasets, recent models have reached a performance bottleneck. Large-scale and high-quality training examples are crucial for training a robust math solver, but existing high-quality datasets have limited scale, and annotating or synthesizing vast MWPs explicitly is highly expensive. To address these issues, we propose a novel hidden space-based retrieval augmentation self-distillation method, named RASD, to improve the mathematical reasoning performance of MWP solvers with semantic representation augmentation and self-distillation learning. RASD enhances problem representations by retrieving and merging similar ones. It then inputs both the original and augmented representations into the decoder for solution reasoning. A self-distillation objective is used to maintain reasoning consistency between them. Extensive experiments on five popular math word problem-solving benchmarks, including MAWPS, Math23K, ASDiv-A, SVAMP, and GeoQA, show the effectiveness and universality of our RASD on improving the math reasoning ability of multiple popular baseline solvers.