The application process for medical training fellowships has evolved significantly over the last decade. Important changes in access to quantitative data, the standardized application itself, and an unprecedented shift in the interview process have forced fellowship leaders to alter their evaluation process. This study aimed to assess each of those aspects of the pediatric cardiology fellowship application process. We review the application itself, focusing on the Experiences section of the Electronic Residency Application Service (ERAS) evaluation and the Pediatric Cardiology Fellowship Evaluation Form, which accompanies each letter of recommendation (LOR). In addition to these novel analyses, we present peer-reviewed literature on the topics of transitioning to pass/fail scoring for USMLE Step 1, declining provision of medical school class rank (especially among top-tier institutions), and the abrupt pivot from in-person interviews in favor of a virtual format. To accomplish these goals, we conducted a retrospective, mixed-methods analysis using three primary approaches. We performed a quantitative assessment of word counts in the Experiences section of ERAS applications submitted to a large pediatric cardiology fellowship program from 2014 to 2016 and compared them with those from 2024 to 2025, a decade later. We analyzed the Pediatric Cardiology Fellowship Evaluation Form for 138 applications, encompassing 2,680 individual ratings across six competency domains. Finally, we examined peer-reviewed research on the loss of objective performance measures in medical school and the Step 1 exam as well as literature on in-person vs. virtual interviews. We found a 78% increase in the median number of words per experience entry in the ERAS application, creating an administrative burden for reviewers, especially in large programs. Evaluation Form data revealed dramatic score inflation, with the applicant pool having to be substantially superior to their peers to warrant the scores received. While the data are mixed, there is a large body of literature raising concerns that the removal of Step 1 numeric scores, declining reporting of medical school class rank, and the loss of in-person interview experiences have together compromised educational leaders’ ability to accurately assess applicants' qualifications for their training programs. These findings highlight a disconnect between both quantitative data and face-to-face interactions that could be beneficial for a fellowship selection committee and raise concerns about how changes in application and recruitment practices may affect both leadership and trainees in advanced medical training.
Kyle et al. (Thu,) studied this question.