Exploring LLM AI in Automatic Generation of Abstracts for Research Publications

A well‐prepared abstract can help researchers in finding their needed resources by succinctly presenting main points of the study in the paper. However, it is a time‐ and effort‐consuming task to create a quality abstract, which captures important key points of the full manuscript. In this study, we aimed to explore the possibility of using LLM AI as a tool to support authors who would like to draft a quality abstract for a research paper. We compared semantic similarities of abstracts that were prepared by the authors, generated with LLM AI, and the full‐text content of 120 papers from ASIS&T 2024 conference. Findings include that different prompt engineering techniques did not generate semantically different abstracts, meaning that the baseline prompts performed well possibly due to the advancement of LLM AI models. Also, experts preferred AI‐generated abstracts over the authors' abstracts when there was semantic discrepancy between the two types of abstracts. This may indicate the usefulness of LLM AI as a tool to support human authors, who may be struggling to draft a quality abstract of their research manuscript.