Inter-Evaluator Consistency in Evaluation of Midwifery Studentsʼ Records in Clinical Practice for Antenatal Health Check-ups ‒ Effectiveness of a Developed Rubric ‒

Background The main concern of clinical practice in Japanese midwifery education is assisting in deliveries. The national guidelines for midwifery schools explicitly require delivery assistance, but with regard to antenatal care, the national guidelines require students to handle ongoing care from the second trimester through postnatal confinement in at least one case. Therefore, it has been left to the discretion of each midwifery school whether clinical practice includes antenatal health check-ups, and relatively little attention has been given to instruction or standards for evaluating midwifery studentsʼ records. Therefore, there is a need to develop reliable, objective evaluation methodology for evaluating antenatal health check-ups conducted by students in clinical practice.Objectives This study was performed to raise objectivity in the evaluation of midwifery studentsʼ records in antenatal health check-ups in clinical practice by （1） measuring disparities in agreement among evaluators and （2） developing a rubric for designing evaluation sheets to minimize disparities between multiple evaluators due to individual views.Methods1）Research term January ‒ August, 20162）Subjects Six midwifery evaluators that had been in charge of clinical practice in performing antenatal health check-ups, including three from the 2-year midwifery program of University A and three from a graduate program at University B.3）Instruments used for evaluation （1） Evaluation tool developed based on the pregnancy care checklist in the “Nurse Midwife Training Guide” published by the Japanese Nursing Association; （2） a rubric we developed for evaluation of midwifery studentsʼ records from clinical practice in conducting antenatal health check-ups.4）Student records used in the study We examined records from antenatal health check-ups in 10 cases conducted by midwifery students who had completed clinical practice sessions of 2-year midwifery school.5）Analysis method Two assessment instruments were applied in each case of antenatal health check-ups by midwifery students. The six evaluators consisted three from A University and three from B University. Fleissʼ kappa was used to compare the disparities in results between the two groups of evaluators.Results Agreement of evaluatorsʼ student evaluations with the sheet based on Japanese Nursing Association pregnancy care checklist: There was “poor agreement” in three cases （30%） in the University A group and in seven cases （70%） in the University B group. There were no cases of relatively high “moderate” agreement. There was “poor agreement” for 12 evaluation items in the University A group （75%） and for nine items in the University B group （56.3%）. There was “slight agreement” for two items in the University A group （12.5%） and five in the University B group （31.3%）. Agreement of evaluatorsʼ student evaluations with the sheet based on our rubric: “Poor agreement” was observed in one case in the University A group （10%） and four cases in the University B group （40%）. Agreement was “slight” in five cases （50%） in the University A group and four cases （40%） in the University B group. “Moderate” agreement was observed for three cases （30%） in the University A group. There was “poor agreement” for two evaluation items in the University A group （25%） and four in the University B group （50%）. Agreement level was “slight” for five items in the University A group （62.5%） and three items （37.2%） in the University B group.Conclusion To assess evaluatorsʼ objectivity in evaluating midwifery studentsʼ records of antenatal health check-ups in clinical practice, this study examined levels of disparities among evaluations given by evaluators from two midwifery schools. Evaluators from University A showed a 30% rate of “poor agreement” in their evaluations of the same student records, while the rate was 70% for evaluators at University B. Disparity levels were reduced to 10% and 40%, respectively, when evaluation standards based on our rubric were used. These results suggest that introduction of the rubric as an instrument for evaluation of antenatal health check-ups could be an effective means of improving consistency among evaluators with regard to evaluation of student performance.