Student and Teacher Perceptions of a Classroom Response System: Demographic Comparisons in a First Semester Calculus Course

This paper assesses the implementation of formative assessment in a Calculus I course for engineering students with Pearson's Classroom response system (CRS): Learning Catalytics. Formative assessment is an instructional technique in which teachers quickly assess students during class for minimal reward/penalty. This assessment helps both students and teachers to identify knowledge gaps and address them [1]. This can be greatly beneficial to entry-level mathematics courses in engineering programs, where misconceptions often go unnoticed until an important summative exam that is critical for student grades. In fact, some research has already shown that formative assessment benefits students in calculus courses [2]–[4]. Although formative assessment has been used for many years with clickers, recent advancements in technology have made it easier to implement formative assessment in class. Technologies now include phone and computer-based Classroom Response Systems (CRSs) such as TopHat, Kahoot and Mentimeter. Reviews have suggested that CRSs provide benefits for students by increasing engagement and offering instant, formative assessment [5], [6]. However, research on these technologies is limited. Recently, the Classroom Response System Perceptions (CRiSP) questionnaire [7] was developed to assess student perceptions of a CRS's usability and its effect on their engagement and learning. Use of this survey is limited in the literature, but provides very helpful feedback to instructors as they work to add useful formative assessment in their courses. The current study measures student and teacher perceptions of CRS Learning Catalytics as it was used for formative assessment in two sections of Calculus I classes in fall 2019 (N = 140). The implementation of formative assessment was a deliberate modification of a primarily lecture-based Calculus I course, and the CRS was intentionally used during most lectures each week. The lead instructor of the course had 38 years of experience with primarily lecture-style instruction. At the end of the semester, students were given the CRiSP questionnaire, and the lead teacher reflected on the implementation. Our research questions include: RQ1: Did students perceive Learning Catalytics to be usable and effective for engagement and learning? RQ2: Were there differences in perceptions for at-risk student groups (female, or low SES; the racial minority student population in this sample was too small for analysis)? RQ3: How did the instructor perceive the use of the Learning Catalytics? We first used descriptive statistics to evaluate learner perceptions of usability, engagement, and learning using responses to the CRiSP questionnaire. The survey results showed that students perceived Learning Catalytics to have a positive influence on all three components. In the full paper, survey responses will be compared across gender (female, N = 41) and socio-economic status (Pell-Eligible, N = 38) using statistical analyses of variance (ANOVAs), using weighted means if necessary due to unequal sample sizes. Instructor perceptions will also be described.