Developing and evaluating a remote, self-administered, AI-powered occupational therapy initial assessment: A crossover randomized controlled trial

Background Artificial intelligence (AI) is spreading through health care, yet systems that administer assessments directly to clients remain untested in occupational therapy (OT), where the initial assessment both gathers data and builds therapeutic rapport. Objective To compare user experience of a purpose-built, remote AI-administered OT initial assessment with a human-administered assessment. Methods Thirty adults completed an AI-administered and a human-administered OT initial assessment in randomized order one to two weeks apart (crossover design), rating usability, perceived compassion, goal setting and satisfaction after each and stating an overall preference; 28 gave reflection interviews. Formats were compared within participants (Wilcoxon signed-rank, McNemar, Holm-corrected) and interviews analyzed thematically. Results Participants were randomized 15 per sequence; order affected no outcome. AI usability was high (modified System Usability Scale 81.1). Perceived compassion favored the human format (median 9.90 vs 7.53; Hodges–Lehmann difference 2.39, 95% CI 1.56–3.11; p < 0.001), as did satisfaction (all p < 0.001). Agreement that goals were clear, personalized, relevant, attainable and motivating did not differ (all p ≥ 0.25), but strong agreement was more frequent after the human format on four of five items (40.0–53.3 percentage points, all p ≤ 0.002). Overall, 83% (25/30) preferred the human format. Four themes were identified. Conclusions The AI-administered assessment was highly usable and produced goals accepted as clear, relevant and attainable at rates matching the human format; what differed was the depth of endorsement, perceived compassion and preference. Conversational AI appears better placed to supplement therapist-led assessment than to replace it.