Abstract Background: Artificial intelligence (AI) systems are increasingly deployed in primary care to classify complex, non-disease phenotypes such as frailty states, depression risk profiles, cardiometabolic subtypes, and multidimensional health constructs. While reported accuracy is often high, the structural stability of these systems across populations and practice environments remains insufficiently examined. Methods: This editorial draws on patterns observed across a structured synthesis of 35 multimodal deep learning studies applied to a non-disease phenotype framework used in community preventive care (PROSPERO: CRD420261276844). Results: Three recurring concerns emerged: asymmetric sensitivity and specificity, extreme cross-study heterogeneity, and limited external validation. In the reviewed literature, sensitivity ranged from 0.31 to 0.70 across categories, whereas specificity ranged from 0.87 to 1.00. Between-study heterogeneity was extreme, with I2 values frequently exceeding 95%, and the 95% prediction interval for overall accuracy ranged from 0.51 to 1.00. These values are presented as indicators of performance dispersion across contexts rather than as definitive performance benchmarks. Conclusions: Architectural sophistication alone does not ensure cross-context robustness. For primary care—where populations and workflows are inherently heterogeneous—instability may translate into misclassification, inequitable performance, and erosion of trust. We argue for a shift from architecture-centric benchmarking to phenotype-governed development. A four-pillar KNOW framework—Knowledge-grounded phenotype definition, Normative data governance, Outcome-calibrated validation, and Workflow-integrated deployment—is proposed to guide safer and more reliable implementation of phenotype-oriented AI systems in primary care.
Shucheng Chen (Fri,) studied this question.