What we measure when we measure competency: progress, challenges, and future directions

Objective To examine what accumulated measurement evidence establishes about the structure of clinical competence, why current competency assessment practices remain misaligned with that structure, and what would be required to modernise them.Methods Evidence was synthesised from independent research programmes in professional psychology that have measured trainee competency, together with parallel structural and outcome evidence from graduate medical education and surgical training, spanning multiple measures, analytic approaches, and samples.Results Across independent programmes, supervisor ratings recover a hierarchically structured construct organised around a dominant general factor of Competency, which predicts patient outcomes, sitting atop a secondary foundational and functional substructure of specific competencies. Parallel evidence from medicine confirms that this general factor carries the primary predictive link to patient welfare. Persistent problems in competency assessment, including reliance on proxy metrics such as accumulated hours, invalid self-appraisals, and the proliferation of unvalidated forms, trace to a mismatch between current evaluation tools and the actual structure of the construct.Conclusions Modernising competency assessment requires standardising the evaluation stimulus through simulation and algorithmic tools that filter out local programme noise, alongside building a shared normative infrastructure. Because the psychometric tools to measure Competency directly now exist, professional psychology has an ethical obligation to its future patients to retire obsolete proxies and implement validated tracking systems.