Short Text Segmentation for Improved Query Processing

This is the authors' abstract. We don't add key points for this paper.

Understanding short texts is crucial to many applications, but challenges are many. Short texts do not always detect the syntax of a written language. As a result, traditional natural language processing tools, ranging from part-of-speech tagging to dependency parsing, cannot be easily applied. Semantic knowledge is required in order to better understand short texts. In this work, a model for short text understanding which exploits semantic knowledge is used. The results show that semantic knowledge is indispensable for short text understanding, and our knowledge-intensive approaches are both effective and efficient in discovering semantics of short texts. A query data set containing short text is pre-processed by removing punctuation marks, numbers, stop words removal and stemming. Then POS tagging is performed and Co-Occurrence network is formed to obtain the semantic similarity. Term Similarities are calculated and finally clustering is done using K-Means, hierarchical, PAM, CLARA, DIANA and AGNES methods and the result reveals that hierarchical clustering gives optimal clusters.