Mobile application rating scale for health professionals (pMARS) to assess the quality of mHealth applications: questionnaire development and psychometric analysis (Preprint)

BACKGROUND Existing tools for evaluating mobile health apps (MHAs) are not well equipped to assess the quality of apps designed for healthcare professionals (HCPs) from their perspectives. OBJECTIVE Therefore, we developed a new tool based on the Mobile App Rating Scale (MARS), named pMARS, and conducted a psychometric analysis of this tool. METHODS Phase 1 adapted MARS for use as pMARS. We established the judgment validity of pMARS through face and content validity. It also included a qualitative study conducted among HCPs to understand users’ perspectives. Phase 2 invited HCP participants to complete pMARS based on their experience with the LabMed app. We established construct validity through internal consistency (Cronbach’s alpha) and structural equation modeling (SEM) to assess the interrelations between various latent constructs. Finally, we employed item response theory (IRT) to evaluate each item’s impact on latent constructs of interest. RESULTS At the end of Phase 1, pMARS had 26 items across five domains: engagement, functionality, aesthetics, information, and subjective quality. Phase 2 analysis (n=218) established good reliability (Cronbach’s alpha: 0.855-0.931). SEM analysis demonstrated that functionality had the strongest influence on end-user willingness to use/recommend and purchase the MHA. IRT identified customization and interactivity of the engagement domain as having a weak impact on latent concepts, whereas entertainment has a higher impact. Ease of use and gestural design had a weak impact on the functionality domain, while arrangement and size of content and quantity and quality had a strong impact on aesthetics and information domains, respectively. CONCLUSIONS This study introduces a novel tool, pMARS, for evaluating MHAs designed for HCPs, and is the first to use IRT in MARS or its derivatives. We recommend developing dedicated evaluation tools for HCPs and using IRT in future studies to refine pMARS by removing ineffective items. CLINICALTRIAL NA