Stepping up of MCQs’ quality through a multi-stage reviewing process

BACKGROUND: Assessment is a crucial and complex process. Thus quality assurance of assessment Methods is essential when assessment is used for the purposes of certification. AIM: To evaluate the effect of continuous well-structured process of the question bank revision, enlightened by item analysis, in improving the quality of Multiple-Choice Questions (MCQs). SETTING: The Family Medicine (FM) training certification exam for the Egyptian Board accredited for International Membership of the Royal College of General Practitioners (MRCGP[INT]). METHOD: The results of the item analysis of Applied Knowledge Tests (AKTs) of two academic years (2009, 2013) were included in the study. The AKT test consisted of two papers, each of 100 MCQS, and blue printed against the FM training curriculum. A total of 226 candidates sat for the two exams; 102 in 2009 and 124 in 2013. RESULTS: There were more MCQs with moderate difficulty and higher discrimination in 2013. Significant improvement was found in the discrimination index (DI) values in 2013 (p < 0.001). and questions with a high facility and classified as easy decreased from 40.5 to 28.7%. The average number of functioning distractors per item increased from 1.99 in 2009 to 2.19 in 2013 (p = 0.015). CONCLUSION: Revision of well-constructed MCQs on a regular basis, and in a structured manner, improved the quality of the MCQs and consequently improved the validity of the examination.