Pilot study shows pharmacy students used the FLUF Test for evaluating AI responses to drug information, indicating the framework’s effectiveness and the need for human input.
As generative artificial intelligence (AI) tools become increasingly integrated into healthcare, health professions education must adapt to ensure students can critically evaluate AI-generated information. To address this need, a pharmacy school assignment was revised to include the critical evaluation of AI-generated outputs in response to a drug information question. The assignment aimed to educate students about AI and equip them with the skills to assess AI-generated information using the "Format, Language, Usability, and Fanfare (FLUF) Test" framework. This pilot study analyzed how twenty-one third-year pharmacy students applied the FLUF Test to identify issues in AI-generated responses to a drug information question and explored their perceptions of the framework’s usability through an open-ended survey. Data was analyzed using descriptive statistics and thematic analysis. FLUF Test results demonstrated that students most frequently identified usability concerns, particularly credibility, in the AI-generated responses. Thematic analysis of students’ open-ended survey question comments revealed three themes: the FLUF Test was useful, easy to use, and the students recognized the need for the human element when interacting with AI. However, some students noted that applying the FLUF Test could be subjective. This pilot study demonstrates the potential value of using the FLUF Test to aid in AI content evaluation in pharmacy education and theoretically other areas of instruction.
No takes yet. Share an insight, caveat, or question.
Alexander et al. (2025) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: