Large language models (LLMs) are transforming how biotechnology review articles are conceived and written. This study evaluates LLMs in generating scientific review articles across three case studies with the focus on biomanufacturing and microbiology. Structured prompts were used across models under "Deep Research" modes, and both statistical and manual analysis (e.g., writing structure, citation metrics, critical depth, citation validity, and hallucination rate) were conducted. Results revealed that while LLMs can summarize large volumes of literature and generate structured outputs with coherent flow and illustrative tables, they fell short in critical analysis, quantitative reasoning, and citation reliability. "Pro" versions of LLMs produced more accurate and extensive citations than base versions, though issues of redundancy, bias toward specific Open Access publishers (e.g., MDPI, Frontiers), and occasional fabrication persisted. Among models, GPT-5 Pro was deeper in its analysis but had fewer citations. Gemini 2.5 Pro and the Perplexity Pro search engine retrieved broader literature with limited critique. Qwen 3 Max and DeepSeek R1 offered moderate balance, with the latter having higher hallucination rates in the application programming interface (API) version. Overall, "Pro" versions of LLMs generate decent summaries of reviewing topics but lack scholarly standards of novelty, rigor, and citation accuracy. Strategic integration and cross-validation of LLMs improve the quality of review papers. The findings underscore the need for new opinions, quantitative analysis or simulation, and interdisciplinary knowledge synthesis to publish valuable biotechnology reviews.
Muddana et al. (Tue,) studied this question.