What does this research mean for the field?

GPT-4o provides lifestyle guidance for atrial fibrillation patients with comparable scientific consensus to physicians and has a lower error rate with greater helpfulness. Novelty: ClaimNovelty.NOVEL_FINDING. Consensus alignment: ConsensusAlignment.SUPPORTS_CONSENSUS.

What question did this study set out to answer?

Assess the clinical utility of generative AI in providing personalized lifestyle guidance for atrial fibrillation patients.

January 10, 2026Scientific Reports0 citationsOpen Access

Evaluating the appropriateness and safety of generative AI in delivering lifestyle guidance for atrial fibrillation patients

MMMasahiro MakinoKyoto Prefectural University of Medicine WSWan Jou SheKyoto Institute of Technology PSPanote SiriarayaKyoto Sangyo University

Key Result

GPT-4o provided lifestyle guidance for atrial fibrillation patients with comparable scientific consensus to physicians and had a lower error rate with greater helpfulness.

Key Points

Assess the clinical utility of generative AI in providing personalized lifestyle guidance for atrial fibrillation patients.
Evaluated three AI models: GPT-4o, DB GPT, and PubMed GPT.
Conducted assessments on guidance from AI models versus traditional physician counseling.
Analyzed responses to sixty-six questions categorized into various lifestyle domains.
GPT-4o matched electrophysiologist consensus levels but with lower error rates.
DB GPT and PubMed GPT maintained similar error rates and empathetic responses to physician guidance.
AI models showed varying strengths in providing specialized content for exercise and diet.

Structured PICO

Does generative AI (GPT-4o, DB GPT, PubMed GPT) provide appropriate, safe, and empathetic lifestyle guidance compared to an electrophysiologist for patients with atrial fibrillation?

Population

16 patients with atrial fibrillation (mean age 69.4, 81.3% male, 62.5% paroxysmal AF) who generated 66 lifestyle-related questions.

Intervention

Three Large Language Models (LLMs): GPT-4o, DB GPT (retrieval-augmented model using curated Q&A), and PubMed GPT (modular RAG model retrieving from PubMed).

Comparator

Responses provided by an experienced electrophysiologist.

Outcome

Evaluation of responses across 10 dimensions (Scientific Consensus, Extent of Possible Harm, Evidence of Incorrect Comprehension, Evidence of Incorrect Retrieval, Evidence of Incorrect Reasoning, Inappropriate Content, Incorrect Content, Specialized Content, Bedside Manner, and Helpfulness of the Answer) by 5 independent blinded electrophysiologists.

Generative AI, particularly GPT-4o, can provide highly empathetic, helpful, and scientifically accurate lifestyle guidance for atrial fibrillation patients, performing comparably or superiorly to electrophysiologists in several domains.

Main Result

Absolute Event Rate: 0% vs 0%

Abstract

Abstract Lifestyle factors play a major role in atrial fibrillation (AF) incidence, but the effectiveness of lifestyle counseling varies among individuals. Due to limited consultation time, physicians often provide only brief guidance, leaving patients to manage changes on their own. This study assessed the clinical utility of three Large Language Models (LLMs) for delivering accurate and personalized lifestyle guidance: (1) GPT-4o, (2) a retrieval-augmented model using a curated Q&A database (DB GPT), and (3) a modular RAG model retrieving evidence from PubMed (PubMed GPT). Sixty-six questions from 16 AF patients were categorized into exercise, diet, lifestyle, and other domains. Five experienced electrophysiologists independently evaluated LLM-generated lifestyle guidance and physician-provided counseling responses using ten dimensions. GPT-4o demonstrated a comparable level of scientific consensus to electrophysiologists, while achieving a lower error rate and significantly higher levels of specialized content, empathy, and helpfulness. DB GPT and PubMed GPT showed similar error rates, proportions of specialized content, empathy, and helpfulness compared to electrophysiologists, but exhibited strengths in specialized content in exercise-related and accuracy in diet-related dimensions. These findings suggest that integrating complementary model strengths may help develop safer and more reliable medical AI systems.

Ask AI

Helpful

Bookmark

View Full Paper