Revisiting rainfall to explore exam questions and performance on CS1

The Rainfall problem comprises small tasks that have been used to investigate student performance in introductory programming. We conducted several kinds of analyses to inform our understandings of student performance in CS1 relating to this problem. We analyzed implementation approaches and program errors, as in related studies, and also explored the role of test writing vis-à-vis the most common student error. Finally, using correlation analyses and manual inspection of the exam answers, we studied how well the Rainfall problem served as an exam question. The students' implementation choices reflected their familiarity with particular loop constructs, while the single most common error concerned division by zero (DivZ), as in many previous studies. Although many students wrote unit tests to guard against DivZ, they often failed to implement according to the tests, which could be partly attributed to the pen-and-paper exam format. From the correlation analyses, we concluded that some students had difficulties with knowledge transfer and that their conceptual understandings of program constructs were insufficient, regardless of them being able to produce working code. The results inform CS1 teachers in preparing materials and exams.