Improving large language models accuracy for aortic stenosis treatment via Heart Team simulation: a prompt design analysis

Figures

• 

  Figure 1. Different clinical decision-making approaches with large language models. Each rectangle represents a ‘thought,’ a coherent linguistic sequence serving as an intermediate step toward making a treatment decision. Self-consistency relies on repeating the same query multiple times independently and retaining the final decision via majority vote. (A) Without self-consistency; (B) with self-consistency. Adapted from Yao S, et al. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems. 2024;36.

• 

  Figure 2. Study workflow. AI, artificial intelligence; SAVR, surgical aortic valve replacement; TAVI, transcatheter aortic valve implantation.

• 

  Figure 3. Forest plot of the relationship between prompt length and mean accuracy. (A) Prompt length in logarithm (tokens). (B) Prompt length in number of examples for few-shot prompting. Each graph displays a linear mixed model with linear (dashed line) and quadratic (solid line) fits, along with P-values. CoT, Chain-of-Thought; ToT, Tree-of-Thought.

• 

  Figure 4. Bar chart comparing mean accuracy across prompt types and prompting techniques. (A) Shows the mean accuracy for the three types of prompts (0-shot, CoT, and ToT) with P-values for inter-comparison. (B) Displays the mean accuracy of different prompting techniques (free- and guided-thinking, few-shot prompting) on the right, and the overall mean accuracy with and without SC on the left. P-values for inter-technique comparison are included. 0-shot, zero-shot; CoT, Chain-of-Thought; SC, self-consistency; ToT, Tree-of-Thought.