Mp17-01 Development of a Generative AI Framework for Automating Social Media Posts From Urological Journal Articles: A Prospective Study

You have accessJournal of UrologyEducation Research I (MP17)1 May 2024MP17-01 DEVELOPMENT OF A GENERATIVE AI FRAMEWORK FOR AUTOMATING SOCIAL MEDIA POSTS FROM UROLOGICAL JOURNAL ARTICLES: A PROSPECTIVE STUDY Lorenzo Storino Ramacciotti, Severin Rodler, Daniel Mokhtar, Jacob Hershenhouse, David Strauss, Luis G. Medina, Andrew Chen, Mihir M. Desai, Andre Luis Abreu, Inderbir S. Gill, and Giovanni E. Cacciamani Lorenzo Storino RamacciottiLorenzo Storino Ramacciotti , Severin RodlerSeverin Rodler , Daniel MokhtarDaniel Mokhtar , Jacob HershenhouseJacob Hershenhouse , David StraussDavid Strauss , Luis G. MedinaLuis G. Medina , Andrew ChenAndrew Chen , Mihir M. DesaiMihir M. Desai , Andre Luis AbreuAndre Luis Abreu , Inderbir S. GillInderbir S. Gill , and Giovanni E. CacciamaniGiovanni E. Cacciamani View All Author Informationhttps://doi.org/10.1097/01.JU.0001008628.15460.84.01AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: This prospective study assessed a novel Generative-AI framework to automate the generation of compelling and accurate social media posts from urological journal articles. METHODS: A total of 150 tweets from the Journal of Urology, European Urology, and European Urology Oncology journals were retrospectively collected from August 2022 to October 2023, and their abstracts were input into ChatGPT version 4.0. A developed prompt was used to auto-generate social media posts, which included title summarization, key findings in bullet points, pertinent emojis, hashtags, handles, and DOI links to the article. Auto-generated posts were assessed for the achievement of tetrafecta of correctness criteria (accurate title summarization; accurate key findings; emoji appropriateness; hashtag appropriateness). 15 scenarios were created with 5 randomly selected tweets from each journal. Each scenario contained both the original tweet and our GPT-generated post for the same article, abbreviated into 280 characters. Five questions were formulated to investigate the likability, shareability, engagement, understandability, and comprehensiveness of the posts. The paired posts were then presented to blinded Amazon Mechanical Turk (AMT) responders for preference evaluation. Statistical significance defined as p<0.05. RESULTS: Mean (SD) time for post auto-generation was 11 seconds (±3.8). Of the 30 rated GPT-generated posts, (83%) met the correctness tetrafecta independently evaluated by two certified physicians: 30 (100%) accurately incorporated the article title, 30 (100%) correctly presented the articles' main findings, 29 (96.7%) and 25 (83.3%) appropriately used emojis and hashtags, respectively. A total of 411 AMT responders answered the survey. The GPT-generated posts consistently outperformed the original journals' tweets, with GPT-generated posts preferred in all metrics assessed: likability (58% vs 42%, p<0.001), engagement (58% vs 42%, p<0.001), shareability (57% vs 43%, p<0.001), understandability (56% vs 44%, p<0.001), and comprehensiveness (67% vs 33%, p<0.001). CONCLUSIONS: This study shows that Generative AI can automate the creation of social media posts from urological journal abstracts that are both accurate and preferable by the general public. Download PPT Source of Funding: None © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e290 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Lorenzo Storino Ramacciotti More articles by this author Severin Rodler More articles by this author Daniel Mokhtar More articles by this author Jacob Hershenhouse More articles by this author David Strauss More articles by this author Luis G. Medina More articles by this author Andrew Chen More articles by this author Mihir M. Desai More articles by this author Andre Luis Abreu More articles by this author Inderbir S. Gill More articles by this author Giovanni E. Cacciamani More articles by this author Expand All Advertisement PDF downloadLoading ...