Do Students Gather Information to Inform Design Decisions? Assessment with an Authentic Design Task in First-Year Engineering

Information gathering is a very important aspect of the design process, one that is used continuously throughout the project to make informed design decisions. This study reports the development of an authentic instrument used to assess skills related to information gathering in first-year engineering students. Existing assessment tools, such as the scenario-based Self-Knowledge Inventory of Lifelong Learning (SKILLS), developed by Conti & Fellenz, or the Self-Directed Learning Readiness Scale (SDLRS), developed by Guglielmino, were evaluated. While these have the advantage of being very easy to administer and analyze, one significant disadvantage is that they are self-reported and thus assess perceptions instead of demonstrated skills. For this study, we have developed a coding protocol to qualitatively evaluate written artifacts produced by first-year engineering students as part of their semester design project. The coding protocol identifies the classification, appropriateness, and documentation of individual sources used. The protocol also identifies overall strengths and weaknesses in regard to efficacy of information gathering, use of evidence to support arguments, and documentation of information sources across the artifact as a whole. In a sample of 25 randomly selected studentproduced memos from a pool of 263, approximately 35% of the sources cited could not be classified due to poor documentation, 76% of the sources used were electronic sources, of which less than 20% were rated as high quality. These results suggest that students did not seek information from a variety of quality sources and that documentation was a significant weakness. Results from this study inform design instruction in future first-year engineering courses.