This paper presents a comprehensive augmented reality (AR)-based rehabilitation system for upper-limb recovery that integrates AR-assisted art therapy, automated markerless goniometry, and the interval mathematical modeling of rehabilitation dynamics. The proposed platform combines four interconnected subsystems: a Python-based markerless video analysis module utilizing three stationary IP cameras, MediaPipe Pose Landmarker, and Kalman filtering; an AR art-therapy application developed for the Magic Leap 2 headset using Unity/OpenXR; a server-side subsystem implemented in NestJS/TypeScript; and (iv) a physiotherapist-oriented web application developed in React. The primary objective of the study is the real-time automated assessment of shoulder joint kinematics during AR-assisted rehabilitation sessions, including flexion (160–180°), extension (50–60°), and abduction (up to 180°). To describe and forecast rehabilitation dynamics, interval mathematical models based on recurrent difference equations were developed, enabling the prediction of subsequent joint angle values using the previous 3–4 observations. Structural and parametric identification of the interval models was performed using the artificial bee colony optimization algorithm. Experimental validation was conducted on rehabilitation data collected from five patients with different clinical diagnoses, including bursitis, epicondylitis, capsulitis, osteoarthritis, and fracture-related impairments. Under the considered experimental conditions, the proposed approach demonstrated promising predictive performance, with an angular prediction error below 5° and a correlation exceeding 95% between predicted and measured rehabilitation trajectories. The developed system implements a unified rehabilitation cycle of “execution–measurement–prediction–adaptation”, enabling the continuous monitoring of recovery dynamics, adaptive adjustment of rehabilitation scenarios, and estimation of the rehabilitation duration required to achieve target motor outcomes. The proposed approach contributes to the development of intelligent AR-based rehabilitation systems by combining markerless motion analysis, predictive interval modeling, and adaptive art-therapy mechanisms within a single clinical framework.
Dyvak et al. (Wed,) studied this question.