Personalized Predictive Model to Predict Subtask Success of Medication Adherence Technologies for Older Adults With Diverse Capabilities: Development and Internal Validation Study

Abstract Background Older adults frequently experience cognitive, physical, sensory, motivational, and environmental barriers that affect medication management. Medication adherence technologies (MATs) can support adherence, but their usability varies widely depending on individual abilities and device features. Prior research has largely focused on overall adherence or user experience, providing limited insight into feature-level usability challenges. Objective The aim of the study is to develop and internally validate a personalized predictive model to predict the success of MAT subtasks for older adults with diverse cognitive, physical, sensory, motivational, and environmental capabilities. Methods A mixed methods approach was used, incorporating the assessment of impairments using various standardized questionnaires, measurement of usability metrics through cognitive walkthroughs, and one-on-one semistructured interviews. For this study, we used “subtasks” as the representative of features of the devices. A subtask is a discrete, individual action that forms part of a larger task, specifically designed to achieve a step in the overall process. Participants tested between 1 and 7 devices from a selection of 13 devices. The proportion of subtask success was taken as the outcome measure. Predictors included demographic, clinical, cognitive, physical, sensory, motivational, and environmental characteristics. Personalized predictive modeling using cosine similarity and generalized linear models were compared with nonpersonalized and naive models. Model performance was evaluated using mean square error (MSE) through cross-validation and held-out validation. Results A total of 117 participants (mean age 74.6, SD 7.9 years) were recruited, including 96 participants for usability testing and 21 for the validation, all varying in cognitive, physical, sensory, motivational, and environmental abilities. Both personalized ( m =0.25) and nonpersonalized models ( m =1.0) outperformed naive predictions ( m =1.21), demonstrating that subtask-level success can be predicted using routinely measurable demographic and functional characteristics. During cross-validation, personalized models achieved optimal performance at a matching proportion of m =0.25, with MSEs lower than those observed at higher matching levels, although differences compared with nonpersonalized models were not statistically significant (Self-Medication Assessment Tool SMAT: P =.50; Daily Living Tasks Dependent on Vision DLTV: P =.43). In the held-out validation cohort, personalized models achieved MSEs of 0.89 (SMAT-based) and 1.16 (DLTV-based) at m=0.20, whereas nonpersonalized models demonstrated better performance with MSEs of 0.726 (SMAT-based) and 0.815 (DLTV-based). Models incorporating performance-based vision measures (SMAT-based) consistently outperformed those using self-reported vision scores (DLTV-based) across both personalized and nonpersonalized settings. Conclusions This study demonstrates the feasibility of predicting subtask success of MATs in older adults. While personalization showed limited added benefit in this dataset, the subtask-focused model provides clinically meaningful insights to support evidence-informed selection of medication technologies, reduce usability-related medication errors, and improve adherence outcomes.