Measuring Connections: Novel Methods and Findings

Faculty often observe that students' difficulty with connecting knowledge from across classes or domains limits their ability to fully analyze problems and evaluate trade-offs. Knowledge retrieval and transfer can be particularly challenging when students are presented with a new problem context or expected to make connections across disciplines. Problems related to "sustainability" and/or "systems" exemplify the multi-faceted and multi-disciplinary problems that require students (and professionals) to demonstrate cognitive flexibility. The primary goals of our NSF EEC project were to improve students' abilities to apply sustainable engineering design concepts across different problems or design contexts and to improve assessment of learning gains using direct measures. The more specific objectives which guided research activities for the project's duration were: (1) Identify appropriate measures of cognitive flexibility that apply to design and other open-ended engineering tasks; (2) Develop and adapt instructional materials and assessments to measure and help students improve their ability to transfer knowledge to/across sustainable design problems; (3) Explore differences in students' responses to the interventions between different types of engineering programs. Our poster will share methods, findings, and future work related to each of the three objectives. Specifically, we will describe our efforts to define appropriate measures of cognitive effort and flexibility using direct assessments and brain imaging technology. Using neurocognition measures is an underdeveloped approach in engineering education research, particularly for complex problem-solving like sustainable design. For this project, electroencephalogram (EEG) measurements were used to try to understand student performance on different types of sustainability knowledge tasks along with validated self-report measures. Our study showed that overall students' concept mapping tasks resulted in more complex representations of sustainable design than listing tasks without increasing their EEG-measured cognitive load. Students did, however, report perceptions of exerting more mental effort to complete concept maps than lists. Findings were consistent across four different sustainability prompts, although there was individual variation in cognitive load and performance that warrants further investigation. In terms of the application of sustainability knowledge, we will also present a new sustainable design rubric that has been tested by students and faculty at two different institutions. The rubric is appropriate for both formative and summative assessment of student projects and can be completed by students and faculty. Along with the rubric, we will also highlight the validation process we used, which required adaptation of a widely accepted process for investigating the validity of tests and similar instruments. The poster will conclude with key take-aways and impact from this research project. Future work, beyond the scope of this project, could include further investigations into cognitive load measurement for complex tasks and exploration of how individual students' cognitive biases (e.g., a preference for environmental over social sustainability dimension) may be leveraged to enhance design team performance.