Automated Oropharyngeal Dysphagia Assessment with Mask R-CNN and Kinematic Measures

Oropharyngeal dysphagia (OD) is characterised by difficulty swallowing liquids or food, significantly affecting an individual’s quality of life and potentially leading to serious health issues such as poor nutrition, dehydration and pneumonia. Diagnosis typically involves the use of a video fluoroscopic swallowing study (VFSS), a method that, while effective, is expensive, time-consuming and requires expert interpretation. Recent advancements in artificial intelligence (AI) offer a promising alternative for enhancing dysphagia diagnosis by providing a more efficient and accurate solution. In this paper, we propose an AI-based system for diagnosing OD. The system processes multi-frame image data from VFSS videos using mask region-based convolutional neural network for object detection and segmentation. This method is based on a feature pyramid network and a ResNet101 backbone. It calculates five kinematic measures – ring measurement, hyoid displacement, bolus clearance ratio, pharyngeal constriction ratio and peak oesophageal sphincter – to assess the presence or absence of the swallowing disorder. The system was evaluated in real time on 250 patients (150 males and 100 females), classifying them as either with or without dysphagia, and achieved an accuracy of 96.8%. This system is expected to significantly assist clinicians. KEYWORDS Artificial intelligence, fluoroscopic data, multi-frame image, oropharyngeal dysphagia, pyramid network, swallowing disorder