WIP: Effectiveness of Different Reflection Approaches for Improving Mastery in an Engineering Laboratory Course

Providing students with detailed, descriptive feedback and having them reflect on what they have learned can foster self-directed learning (Ambrose, 2013); a critical ability for future engineers whom need to be able to translate their skills and knowledge to novel situations (Bary & Rees, 2016). Standards-based grading (SBG) has been slowly emerging in the engineering education field as a way to provide students with feedback on how well they are meeting course objectives (Carberry, et. al., 2016). This grading approach contrasts traditional summative based grading which only shows students what they got incorrect and fails to provide assessment of the learning objectives with which they struggle. Our previous research investigated SBG implementation to evaluate lab reports in engineering lab-based courses and identified student weaknesses in two standards: problem identification and interpretation (to be added after review, 2020). Work demonstrating improved SBG value with structured reflection (Diefes-Dux & Castro, 2018) motivates us to leverage reflection in developing students' metacognition with the ultimate goal of improving mastery in their weaknesses. For scientists and engineers, laboratory and design notebooks record a project from its start to completion. When done well, these notebooks are an inherently reflective practice on one's own learning, understanding, and decision-making process (Svarosvsky & Shaffer, 2006). We hypothesize that reflection, especially if done while learning (e.g. lab notebooks) in addition to reflection after receiving SBG feedback, will improve mastery in "problem identification" and "interpretation" as well as result in favorable student attitude about SBG and reflective practices. We will test our hypothesis in an introductory experimental design lab course for sophomore Biomedical Engineering students. This course runs twice a year and enrolls ~25 students per offering. In the winter quarter of 2021 (WQ21), we will assign students lab reports (as previously implemented) and add individual structured reflections of their SBG feedback. Students will reflect on what they have learned, areas in which they want to improve, and specific actions they plan to take to improve in those areas (Diefes-Dux, 2016). In addition to the assigned assessments from WQ, students enrolled in SQ21 will be asked to maintain a laboratory notebook. The students will record and reflect on steps taken during the lab, mistakes made, how they fixed their mistakes, etc.—a reflection on their learning while doing. We plan to determine if there are differences in mastery (quantified by SBG of lab reports) across different reflection implementations (no reflection, SQ20; post-assignment reflection, WQ21; reflection while doing + post-assignment reflection, SQ21). In addition, we will assess attitude adapted from Carberry, et. al., 2013, engagement with the process of reflection and SBG based on Diefes-Dux & Castro, 2019, and the quality of reflections similar to Menekse, et. al., 2011 across terms with different reflection approaches. This study will provide insight into how the integration of reflection (i.e. post-assessment vs during assessment) affects mastery of standards, quality of student reflections, and student attitude toward SBG. Ultimately, providing engineering students' optimized opportunities to reflect on their learning may aid their development as self-directed learners.