Assessing Real-Life Food Consumption in Hospital with an Automatic Image Recognition Device: a pilot study

Abstract Background and aims Accurate dietary intake assessment is essential for nutritional care in hospitals, yet it is time-consuming for caregivers and therefore not routinely performed. Recent advancements in artificial intelligence (AI) offer promising opportunities to streamline this process. This study aimed to evaluate the feasibility of using an AI-based image recognition prototype, developed through machine learning algorithms, to automate dietary intake assessment within the hospital catering context. Methods Data were collected from inpatient meals in a hospital ward. The study was divided in two phases: the first one focused on data annotation and algorithm’s development, while the second one was dedicated to algorithm’s improvement and testing. Six different dishes were analyzed with their components grouped into three categories: starches, animal protein sources, and vegetables. Manual weighing (MAN) was used as the reference method, while the AI-based prototype (PRO) automatically estimated component weights. Lin’s concordance correlation coefficients (CCC) were calculated to assess agreement between PRO and MAN. Linear regression models were applied to estimate measurement differences between PRO and MAN for each category and their associated 95% confidence intervals. Results A total of 246 components were used for data annotation and 368 for testing. CCC values between PRO and MAN were: animal protein sources (n= 114; CCC = 0.845, 95% CI: 0.787-0.888), starches (n= 219; CCC = 0.957, 95% CI: 0.945-0.965), and vegetables (n=35; CCC = 0.767, 95% CI: 0.604-0.868). Mean differences between PRO and MAN measurements were estimated at -12.01g (CI 95% -15.3, -8,7) for starches (reference category), 1.19 g (CI 95% -3.2, 5.6) for animal protein sources, and -14.85 (CI 95% -22.1, -7.58) for vegetables. Conclusion This pilot study demonstrates the feasibility of utilizing an AI-based system to accurately assess food types and portions in a hospital setting, offering potential for routine use in clinical nutrition practices.