Using Design-based Research Methods to Scale in an Expanding Intervention

In this work-in-progress paper, we present design-based methodological work intended to support implementation of a large-scale, ongoing NSF Scholarships in STEM (S-STEM) project. We have created pilot tools and procedures for data collection and analysis that will supply consistent research throughlines during the project lifespan while also providing iterations of formative feedback about participants’ needs and their experiences in the project. While these tools and procedures have allowed us to analyze pilot data, in each year of the project, participant numbers will continue to grow, placing additional demands on our research team and our chosen methods. Will the designs we have created help us to scale effectively while remaining fidelitous to our project goals? The purpose of the S-STEM project is to connect student pathways at state technical colleges to Engineering and Computer Sciences programs at a Research I university in the southeastern United States. Toward this goal, we are implementing communities of practice and cognitive apprenticeship strategies to support student cohorts and create programming aspects to enhance transfer students’ enculturation to the university, completion of STEM-related degrees, and placement in the industrial workforce. Cohorts begin at the technical colleges, guided by a doctoral student mentor who engages program participants in applied research and shepherds them throughout their transition to the university. Now in the second year of the project, the pilot cohort is studying at the university while new cohorts are engaged at the collaborating technical colleges. Each year, the number of students participating in the five-year implementation will accumulate to a proposed total of over 300 students. Our research team will need to scale our efforts toward project fidelity. Our intent with this work-in-progress paper is to share our current status and invite interested colleagues to provide feedback about our pilot analysis work and our plans for future data. We will introduce the design-based methods that inform both research and development in our project. In particular, we will focus on the applicability of these methods for implementation work and how they can be effectively scaled to large interventions. We will describe observation, interview and focus group, and survey methods, along with how these methods complement academic sources and other student-related data. As the primary researchers on the S-STEM project, we are also relatively new to ASEE, coming from backgrounds in Mathematics Education and Learning Sciences. We are eager to learn from ASEE colleagues about how we can improve our knowledge and skills as researchers to support work in Engineering Education.