Research topic evolution: a comparative analysis of human and machine approaches

Exploring patterns in large text corpus is essential for effective knowledge discovery in research domains. However, machine-driven methods often introduce noise and rely heavily on parameter thresholds. Human expertise is therefore essential for ensuring reliable outcomes. This study conducts a comparative analysis of a classification task performed by both human and computer algorithms. During the task, human experts are asked to categorize a list of abstracts based on their semantic contents, where computer algorithms perform computations, including network analysis and document embeddings, to group the abstracts. The findings show a significant level of disagreement between human and computer-generated clusters, indicating the need for further investigation into the factors influencing community categorization and incorporating more advanced techniques to improve the results.