Validity of the Contextual Competence Scale for Engineering Students

Abstract Background Engineering educators and practitioners increasingly value contextual competence. A psychometrically sound, practical, and useful scale for assessing undergraduate engineering students' contextual competence is needed. Purpose/Hypothesis This article provides comprehensive evidence of the content, structural, discriminant, and criterion‐related validity of the contextual competence scale. Design/Method This study used student, alumni, and faculty survey data from a nationally representative sample of 120 U.S. engineering programs from 31 four‐year institutions. Validity evidence was obtained by expert review of questions, a pilot test, factor analyses, and several analyses utilizing t ‐tests, correlations, and regression. Results Experts constructed the questions used in the scale (content validity). Those questions combined to measure a single concept (structural validity), the scale reliably measures (Cronbach's alpha = .91) that concept, and it focuses on contextual competence instead of other student outcomes (discriminant validity). The contextual competence scale varies according to students' characteristics and curricular experiences as well as similarities and differences in student and alumni levels of contextual competence in the same programs and institutions (criterion‐related validity). Finally, the scale may be a more accurate measure of contextual competence than faculty members' perceptions of students' ability. Conclusions The contextual competence scale allows engineering programs to meet ABET and other self‐study requirements or do large‐scale research with relative ease and little expense. The process described in this article can be used by other researchers in engineering education for their scale development efforts.