Semantic processing for Urdu: corpus creation, parsing, and generation

Abstract Discourse representation structure (DRS), a formal meaning representation, has been used for both semantic parsing and natural language generation tasks and gained promising results for high-resource languages such as English and for the lesser-resourced European languages Italian, German, and Dutch. We investigate how we can employ DRS for the low-resource language Urdu for neural semantic parsing (translating Urdu sentences into formal meaning representations) and natural language generation (generating Urdu sentences from formal meaning representations). There are no annotated corpora for Urdu available, so we adopted a combined approach involving both manual annotations and rule-based procedures to transform English-aligned DRS into Urdu-aligned DRS through syntactic structure and word surface alignment, because word order in Urdu (subject–object–verb) differs from that of English (subject–verb–object). To further increase the amount of semantically annotated data, we developed lexical, grammatical, and named entity-based augmentation techniques. This resulted in an increase of nine times more data examples. Using the augmented meaning bank for Urdu, we developed a neural semantic parser and generator that benefited significantly from the augmented data and showed more generalization ability compared to the model without augmentation. We evaluated the effect of semantic data augmentation using a transformer-based state-of-the-art neural sequence-to-sequence architecture. Our implementation shows promising results for the semantic processing of Urdu and demonstrates that data augmentation increases performance (F1-Score) for semantic parsing from 67.12 to 76.81, and leads to substantially increased BLEU, BERT-Score, METEOR, ROUGE, and chrF scores for generation.