The Student Attitudinal Success Inventory III (SASI III): Construct Validity and Measurement Invariance

The retention of engineering students, from admission to graduation, is an international concern in engineering education for a long time. Increasing the retained engineering students can potentially possible to increase the number of engineering graduates. Thus, it is crucial to predict or identify students with propensities to drop out of an engineering program, particularly for first-year undergraduate students. Engineering programs admit students with influential cognitive factors, such as grade point average (GPA) and standardized test scores (e.g., SAT and ACT scores), to predict their academic success in university settings. However, non-cognitive factors have been identified as evidence for students' retention and academic achievement. The Student Attitudinal Success Instrument was developed to quantify non-cognitive characteristics of first-year engineering students before entering colleges or universities. The original SASI consists of 161 items assessing nine specific non-cognitive constructs. Later on, five more non-cognitive constructs have been added to the SASI II. After the first development of SASI in 2004, the instrument has been utilized as an important measure to model student retention in the engineering education. As evidence accumulated, the third version of SASI, namely SASI III, consists 140 items quantifying 16 latent constructs. Therefore, it is necessary to validate the SASI III with modified items, because modifications in factors and items potentially change the original psychometric properties. In addition to differences in cohorts, students take SASI III twice during their first semester. There are two time-related grouping variables: cohorts (2018 vs. 2019) and occasions (pre -survey vs. post-survey). Whenever psychometric properties are interpreted, it is critical to examine whether the measurement model tests the hypothesis that similar interpretation can be derived from the data across groups. Unfortunately, previous studies fail to consider grouping variables when validating SASI or SASI II. Thus, this study aims to 1). validate the SASI III with the new set of constructs and items, and 2). examine whether the inventory performance the same across cohorts and occasions. To accomplish these goals, the following research questions are proposed: 1. What level of reliability for each construct in the SASI III overall, over cohorts and occasions? 2. What is the evidence of construct validity of the SASI III, overall, over cohorts and occasions? To answer research question one, the internal consistency reliability analysis is conducted for each construct. The index Cronbach's α is reported for the overall data set, each cohort, each occasion, and both cohort and occasion. To answer research question two, a confirmatory factor analysis (CFA) and multiple groups CFA (MG-CFA) are conducted to examine the established factor structure of the SASI III. The goal for CFA and MG-CFA in this study were to 1). provide evidence of the construct validity of the SASI III as a whole, 2). test a hypothesized factor structure and evaluate whether the same general factor structure of the SASA III is supported in cohorts and occasions.