Using Generalizability Theory to Assess the Reliability of Student Ratings of Academic Advising

The coefficient alpha procedure has been used frequently to assess the reliability of student ratings of academic advising, despite the fact that in many cases, coefficient alpha is not an appropriate procedure for providing reliability information on this type of measurement data. Data were collected from 15 postsecondary institutions’ use of the Survey of Academic Advising, and a more advanced framework, that is, the generalizability theory, was used to analyze the reliability of the data. The results of the study demonstrate that the reliability of student ratings of advising can be estimated more appropriately and accurately by using the procedures of generalizability theory. Advantages and disadvantages of the generalizability theory approach in assessing the reliability of student ratings of advising are discussed in comparison with those of the coefficient alpha procedure.