Work in Progress: A Holistic PhD Admissions Rubric--Design & Implementation

Addressing today's complex engineering challenges requires that all perspectives and lived experiences are brought to the table. The NSF and other organizations have stressed the importance of bringing a more representative proportion of the national population into the professional STEM research pipeline. By investing in building a robust and diverse community of engineers, we will have access to a workforce with the insight into societal needs needed to solve today's problems A comprehensive graduate student recruitment program must focus on increasing the diversity of its applicant pool and matriculating cohorts. Critical to this goal is the use of evidence-based best practices during the graduate admissions process. The admissions process should ensure that every student comes into the school with the same potential for success, regardless of background and experiences. It is therefore crucial that the process rely on true indicators of graduate school success, such as research interest and potential, motivation and persistence, and potential for leadership; while de-emphasizing poorer predictors, including traditional academic preparation indicators and GRE scores. To this end, a Holistic PhD Admissions Rubric was developed and implemented for use by all graduate programs in the engineering school of a large R1 University. This rubric is an evaluation tool built into the central admissions system that aids faculty reviewers through best practices in PhD admissions. The rubric facilitates the use of these practices while still permitting a degree of uniformity to the review process, albeit one that further ensures excellence, access, and equity. The rubric utilizes nine criteria, including metrics such as letters of evaluation scores, demonstration of research potential/vision, evidence of motivation/persistence/ability to overcome obstacles, and potential for leadership. Each criterion includes suggestions on where in the application materials to seek out that information, as well as descriptors for each criterion's numerical rating. A score is given for each criterion and then all criteria scores are totaled and used as part of the overall admissions evaluation. Faculty dialogues, meetings, and educational emails were used to increase implementation of the rubric across the school beginning with the admissions cycle for students matriculating in the Fall of 2019. The rubric was updated for the admissions cycle for students matriculating in the Fall of 2020 based on continued feedback of faculty users, and review by premier experts in the field of holistic graduate admissions. In the admissions cycle for students matriculating in the Fall of 2020, 57% of admitted PhD students had a completed rubric in their application file, and we expect this number to continue to increase. Our research team plans to conduct a survey of the faculty who review PhD applications and a series of focus groups with graduate program admission directors in order to better understand the role the rubric takes in the admissions process. We want to understand how faculty are using the rubric and whether they find it helpful and easy to use. Additionally, we want to know if the rubric has changed how faculty approach the PhD application evaluation process and the value they place on various application metrics. We will also assess faculty perspective on the quality of student being admitted into their programs both before and after implementation of the rubric. This review of faculty perspective will then be compared to admissions data in the two years prior to rubric implementation and two years after, concentrating specifically on GRE scores and GPA, as well as demographic data. We will also assess the first year GPA and first year retention rate of the class matriculating in the Fall of 2019 (the first year of rubric implementation) as a work-in-progress. Future work will focus on longer-term retention rates, as well as other metrics of graduate school success, including time-to-degree and research productivity.