Answers are grounded in this paper and cite their sources.
Computational study demonstrates robust language-independent lemmatization across diverse languages using random forest classifiers, indicating a flexible alternative to rule-based models.
Lemmatization is a process of finding the base morphological form (lemma) of a word. It is an important step in many natural language processing, information retrieval, and information extraction tasks, among others. We present an open-source language-independent lemmatizer based on the Random Forest classification model. This model is a supervised machine-learning algorithm with decision trees that are constructed corresponding to the grammatical features of the language. This lemmatizer does not require any manual work for hard-coding of the rules, and at the same time it is simple and interpretable. We compare the performance of our lemmatizer with that of the UDPipe lemmatizer on twenty-two out of twenty-five languages we work on for which UDPipe has models. Our lemmatization method shows good performance on different languages from various language groups, and it is easily extensible to other languages.
No takes yet. Share an insight, caveat, or question.
Akhmetov et al. (2020) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: