Education and Evaluation for the NRT: Accounting for Numerous Requirements, Multiple Disciplines, and Small Cohorts

The purpose of this paper is to describe our approach to program evaluation for the National Science Foundation National (NSF) Research Traineeship (NRT) Program in Sustainable Food, Energy, and Water Systems at X University. The NRT program is designed to educate and train next generation of engineers, agronomists and scientists to meet local food, energy and water management needs with solar energy. This requires educating interdisciplinary cohorts of graduate students in basic scientific and technical principles to enable them to develop economically viable, innovative, interdisciplinary solutions. This is achieved through many interdisciplinary courses, research experiences and professional development requirements. Assessment and evaluation activities, designed to measure impacts on training and career development, are important aspects of this work, but require significant attention to capture the range of activities undertaken by very small cohorts of interdisciplinary students and faculty. Our goal was to develop "sustainable" evaluation activities given our observation that programs often begin with very ambitious assessment and data collection goals in the first year, that diminish over time. To do so, we addressed challenges associated with startup and operational sustainability, such as: • Tracking individual graduate student participation in various NRT program requirements can be labor intensive and is often done manually (e.g., combinations of disciplinary courses, interdisciplinary courses, professional development modules, and scholarly activities). • There are few existing validated instruments for assessing interdisciplinary training programs, and limited financial support for PI/co-PI involvement in instrument development. • While external evaluators serve an important role in NSF training grants, when programs are in startup mode, they are often far removed from subject matter experts and programmatic offerings. • Response rates can be low when busy students are expected to complete course and program surveys that go beyond evaluations mandated through the university. To overcome these challenges, we optimized evaluation activities to increase compliance and effectiveness through several activities that will be discussed in this paper. These include the creation of a professional development tracking "hub" within our university's learning management system (LMS); the development and dissemination of an annual survey to track various learning and career outcomes over time; interdisciplinary course evaluations; and annual focus group interviews. This paper will report on the rationale for and results of these education and evaluation activities.