Semantic Textual Similarity with Constituent Parsing Heterogeneous Graph Attention Networks

Semantic Textual Similarity (STS) serves as a metric for evaluating the semantic symmetry between texts, playing a pivotal role in various natural language processing (NLP) tasks. To facilitate the accurate measurement of semantic symmetry, high-quality text representation is essential. This paper studies how to utilize constituent parsing for text representation in STS. Unlike most existing syntax models, we propose a heterogeneous graph attention network that integrates constituent parsing (HGAT-CP). The heterogeneous graph contains meaningfully connected sentences, verb phrase (VP), noun phrase (NP), phrase, and word nodes, which are derived from the constituent parsing tree. This graph is fed to a graph attention network for context propagation among relevant nodes, which effectively captures the relations of inter-sentence components. In addition, we leverage the relationships between verb phrases (VPs) and noun phrases (NPs) across sentence pairs for data augmentation, which is denoted as HGAT_CP(NP, VP). We extensively evaluate our method on three datasets, and experimental results demonstrate that our proposed HGAT_CP(NP, VP) achieves significant improvements on the majority of the datasets. Notably, on the SICK dataset, HGAT_CP(NP, VP) achieved improvements of 0.39 and 1.84 compared to SimCSE-ROBERTa_large and SimCSE-ROBERTa_base, respectively.