Key points are not available for this paper at this time.
Methodological reviews have been used successfully to identify research trends and improve research practice in a variety of academic fields. Although there have been three methodological reviews of the emerging field of computer science education research, they lacked reliability or generalizability. Therefore, because of the capacity for a methodological review to improve practice in computer science education and because the previous methodological reviews were lacking, in this dissertation, a large-scale, reliable, and generalizable methodological review of the recent research on computer science education was conducted. The purpose of this methodological review was to have a valid and convincing basis on which to make recommendations for the improvement of computer science education research and to promote informed dialogue about its practice. After taking a proportional stratified sample of 352 articles from a population of 1,306 computer science education articles published from 2000 to 2005, each article was coded in terms of their general characteristics; report elements; research methodology; research design; independent, dependent, and mediating/moderating variables examined; and statistical practices. A second rater coded a reliability subsample of 53 articles so that estimates of interrater reliability could be established. The major findings were that (a) the majority of investigations were insufficiently controlled to make generalized causal inferences, (b) there were no differences in the methodological quality of articles published in journals or those published in conference proceedings, and (c) there was a decreasing yearly trend in the number of articles that only presented anecdotal evidence and the number of articles using explanatory descriptive (e.g., qualitative) research methods. Also, (d) it was found that the region of the first author's affiliation covaried with proportion of articles that reported on experimental or quasi-experimental investigations, explanatory descriptive investigations, and on proportion of articles in which attitudes were the sole dependent measure. In addition, several differences in research practices across the fields of computer science education, educational technology, and education research proper were found.
Julnes et al. (Sun,) studied this question.