Considerations for assessment, evaluation, and continuous improvement of a pre-college STEM summer program for promising Black high school students

Abstract pre-college, race/ethnicity, gender, broadening participation in engineering and engineering technology The purpose of this paper proposal is to present considerations of a preliminary program evaluation of a two-week summer, residential, program for rising junior and senior high school students who identify as African-American or Black. This program offers an opportunity to explore engineering at a high-research university, while also learning about college life. Living in a residence hall, connecting with faculty, and working on an industry-sponsored design project are facets of the program. The coordinators of the program were intentional about the need for culturally responsive activities, training, and opportunities that related to African-American/Black culture and experience. Intentional events focused around music, sports and cultural community engagement activities. Wellness workshops were integrated to highlight important topics such as imposter syndrome, financial literacy, and mental health. The students were also exposed to collegiate level organizations such as the National Society of Black Engineers (NSBE), Society of Women Engineers (SWE), Black Student Alliance (BSA) which are in place to provide African-American and Black students with social and professional opportunities. Along with the intended outcomes of the program, those that are outside of stated outcomes can occur that lead to additional considerations which impact the experience of program participants. Because the structural components of the program were based on a pre-existing summer program for women, it was critical to explore necessary changes to the program based on feedback and the expertise of one of the coordinators' expertise as a STEM high school educator. The authors present a plan for program evaluation and tracking, lessons learned, and future directions. Particularly, a logic model and needs analysis led by the coordinator to address such issues as the need for increasing recruitment in the program for women and insight in identifying students in the classroom environment who are ideal participants. Additionally, identifying outcomes that extend past the summer experience that may arise and be identified through relationships with K-12 educators as outcomes that have impacted interest and efficacy for attending college and engagement in STEM subject matter. This paper aids in the exploration of how the residential summer experience had unintended outcomes based on participant feedback, in the form of focus groups and pre/post survey methods, as well as considerations for the lost years of the pandemic. This program offers an opportunity to explore the many facets of engineering. However, a multicultural perspective places emphasis on Black identity and community building. Participants' lived experiences varied greatly across factors of high school type, academic history, geographic region, and college attendance of family. The students interacted in this space and were able to understand and relate to each other regardless of their academic differences. During the two weeks of the program, students participated in numerous activities with the goal of exposure to a faculty, student, and professional network. This paper is a stepping stone for further program evaluation that serves the voices of the participants involved.