Validation of the ICO-OCEX tool for competency-based assessment in ophthalmology residency programs

Competence in medical training requires trainees to independently demonstrate practical skills. The ICO-OCEX (International Council of Ophthalmology-Ophthalmic Clinical Evaluation Exercise) tool, an improved version of OCEX (Ophthalmic clinical evaluation exercise), employs behaviorally descriptive milestones based on the Dreyfus model to address inconsistent grading interpretations and effectively assess ophthalmology residents’ competencies through workplace-based assessments. To determine the validity and reliability of ICO-OCEX formative assessment tool for ophthalmology postgraduate residents. A multiphase mixed-method instrument developmental approach was used. Quantitative content validation was conducted using the modified Delphi technique, while cognitive pretesting assessed response process validity. Pilot testing was carried out to evaluate construct validity, as well as reliability and inter-rater reliability. The modified Delphi technique achieved consensus on all 33 items, confirming stability and a scale content validity index of 0.97. Cognitive interviews led to revisions, resulting in a 34-item instrument. Confirmatory factor analysis indicated a good to excellent model fit. Cronbach’s alpha was 0.995 and Fleiss Kappa showed moderate to substantial inter-rater reliability. The ICO-OCEX tool, consisting of 34 items measuring four constructs on a three-point Dreyfus scale, demonstrated acceptable validity and reliability. It may be useful for formative assessment and supporting competency development in ophthalmology residency programs, particularly when faculty receive training for consistent application.