Adversarial Analysis and Methods for Math Word Problems

Present-day state of the art models can perform well on most language tasks. Math word problems are at the intersection of linguistic semantics and quantitative logic. Two salient state of the art methods to solve math word problems are evaluated against adversarial examples employing extraneous information, associative reordering and defined relationships. The degradation in models' performance is presented and analyzed in detail. Additionally, proposed methods using quantity cell filtering and semantic mapping are evaluated against adversarial examples. The severe 30%+ degradation in performance and modest improvements using mitigation methods establish a strong need to both build bigger datasets as well as models that can more robustly handle adversarial inputs.