PROTOCOL: Collaborative Testing for Improving Student Learning Outcomes and Test‐Taking Performance in Higher Education: A Systematic Review

This is the authors' abstract. We don't add key points for this paper.

The primary purpose of assessment in an instructional setting is twofold: to determine whether learners have achieved the stated objectives and learner outcomes described in the curriculum and to determine whether educators meet those learning objectives in the classroom. Assessment, quite often, is administered in the form of a written or computerized test or exam. An exam is a measurement instrument designed to measure knowledge and understanding of defined content (Gaberson, 2008). Testing is an important activity for the learner as the learner needs to identify how they performed on the test, and whether test results allow them to progress in their program (Sainsbury & Walker, 2008). Testing is also important for the educator. Testing is a means to identify whether teaching is effective and how well the student comprehends the material. Because testing is a “high-stakes” (Hoachlander, 1998) activity in which students must offer their best performance, it can be anxiety provoking and can negatively affect their performance. There are many reasons why students might underperform on tests, including poor performance related to test anxiety, poor study skills, improper test preparation, language difficulties, and the presence of cultural bias through poorly written questions (Lusk & Conklin, 2003). There is little empirical evidence available on the validity of various testing formats, yet educators still rely heavily on traditional individual testing as a method of evaluation or assessment (Halstead, 2007). An additional complicating factor is that all tests have error; without conducting a proper item analysis, poorly constructed or unvalidated tests can inaccurately reflect student knowledge (Gaberson, 2008). Because of the many disadvantages inherent in traditional testing, alternate testing methods have been explored. One of these is collaborative testing, a relatively recent modality that is becoming more frequently utilized as an alternative. The intervention that will be examined in this review is collaborative testing. Collaborative testing is a method in which students work together while taking a written evaluative exam. Collaborative, or cooperative, testing is an umbrella term used to describe a test method different from traditional or individualized testing. Collaborative testing has been utilized in undergraduate, graduate, and post graduate settings. Collaborative testing, which is a student-centered, active learning approach, is also referred to as group testing, double testing, paired testing, cooperative testing, and dyad testing (Centrella-Nigro, 2012). Group testing is the term utilized when a test is administered to more than two students. Generally, in group testing the group consists of three to six students (Centrella-Nigro, 2012; Leight, Saunders, Calkins, & Withers, 2012; Wiggs, 2011). The terms dyad or paired testing are utilized when students are paired with a partner to take an exam (Centrella-Nigro, 2012; Haberyan & Barnett, 2010; Mitchell & Melton, 2003). Double-testing is another procedural method for administering a group test. In double-testing, a student first takes an exam as an individual and then re-takes same exam either with a partner or in a group. Within each of these methods of collaborative test administration, partner and group selection, as well as time allowed to take the test and methods for grading all vary. The variation of methods and procedures for implementing collaborative testing makes comparisons of collaborative testing outcomes difficult. Collaborative testing is a fairly new concept with little research to clearly demonstrate that this testing method improves student learner outcomes and test taking performance in higher education. (Haberyan & Barnett, 2010; Mitchell & Melton, 2003; Sandahl, 2009; Sandahl, 2010). Active learning is a teaching concept that promotes student engagement. It is often reported that a collaborative learning environment provides more effective learning. Collaborative testing can be a component of active learning where a small group of students actively engage in learning through testing. Cooperative and collaborative learning are subsets of active learning in which students work in groups, teams, or pairs to perform complex tasks. This collaboration encourages active engagement in the learning process as group members discuss test questions, debate and problem solve to determine the best answers and communicate supporting reasons for their particular answer. Because collaborative testing is accomplished by groups of students, the literature often refers to collaborative or cooperative testing as group testing (Jensen, Moore & Hatch, 2002; Sandahl, 2010; Vockell, 1994; Wilder, Hamner & Ellison, 2007). Collaborative testing may provide a mechanism that more accurately evaluates students’ knowledge and understanding of course material. Testing, especially high stakes testing, often causes test anxiety for the student. Test anxiety tends to weaken students’ test taking ability and ultimately their overall grade in a course. Anxiety interferes with the students’ ability to recall knowledge about the material being tested consequently leading to poor test performance (Markman, Balik, Bercovitz & Ehrenfeld, 2010). Collaborative testing, where students’ can look to their peers for answers or for verification of their own answers, can decrease pressure and anxiety allowing improved recall of material and improved test scores. (Mitchell & Melton, 2003). Through active peer collaboration during testing, both high and low-performing students can complement each other's knowledge allowing the combined effort to improve the group's understanding and application of course material (Lusk & Conklin, 2003). Collaboration during testing also alters the concept of competition among the students, encouraging them to work as a team supporting each other's success. The idea that a higher performing student is depending on a lower performing student to contribute to each other's success, may positively impact the lower performing student's attitude about his or her own abilities (Bransford, Brown & Cocking, 2000). Positive self-concepts and test scores ultimately lead a student to improved knowledge and desire to learn and be successful (Phillips, 1988). Analysis of collaborative testing research is difficult for many reasons. Collaborative testing research focuses on a variety of outcomes, including student satisfaction, anxiety, course and end of semester grades, retention of short-term and long-term knowledge, and effects on teamwork. Furthermore, studies reveal multiple methods and procedures for administering and scoring tests that are taken collaboratively. Examples of these variations include differences in group size and group makeup, timing of tests with regards to administration procedure and placement within course, and determination of grades. Collaborative testing research methods also vary widely in the number of students used in the studies, student grade level, type of exam administered (multiple choice, essay, etc.), and type of class or course in which the testing is used. There are debates regarding the advantages and disadvantages of collaborative testing. It has been shown that students perceive that they learn better in collaborative testing and exhibit improved individual test scores (Leight et al., 2012; Sandahl, 2009). Other reported advantages of collaborative testing include, but are not limited to, better critical thinking skills, improved collaboration and team work among peers, enhanced learning and test taking skills, motivation to study material, reduced test anxiety, and improved test taking performance and confidence. Conversely, researchers report that collaborative testing may artificially raise grades for students who otherwise would not pass a course (Jensen et al., 2002; Mitchell & Melton, 2003). Investigators have also demonstrated that collaborative testing does not result in increased learning and retention of course material. (Giuliordori, Lugan, & DiCarlo, 2008; Harton, Richardson, Barreras, Rockloff & Latane, 2002; Leight et al., 2012). Educators describe disadvantages of collaborative testing such as poor effects on student study time and true comprehension of material. Students may not spend the necessary time in individual study if they know that their group may help them to pass the exam. Other disadvantages include lack of student preparation for the group, thus making the overall group score decrease as well as student reports of knowledge insecurity. This insecurity arises from student lack of preparation for the exam, yet passing the exam without truly knowing the material. Internal group conflict is another disadvantage that may negatively impact exam scores as students may be pressured to change originally correct answers. Lastly, long-term retention of material was identified as a disadvantage in that students who tested collaboratively may not ultimately retain the material for a comprehensive individual exam (Lusk & Conklin, 2003; Sandahl, 2009). Several studies have demonstrated that collaborative testing enhanced student learning, validated misconceptions of testing material among peers, and improved test scores. Mitchell and Melton (2003) used collaborative testing for a unit exam on fluid and electrolytes in which students in an Associate Degree nursing program were randomly assigned to groups. The students who volunteered to participate took the exam individually and were allotted 50 minutes for a 50 –item multiple-choice exam. After submitting the individual test, a second test form was given to student groups who were allowed 10 minutes to discuss the questions on the exam and permitted to change their original answers. The second test form permitted an assessment on whether collaboration resulted in a change in the student's original exam grade. Averages of both the individual and group grade were used to determine the final test score. Study authors found through student surveys that collaborative testing helped to validate knowledge and clarify misconceptions. However, they found that collaboration gave unprepared students the opportunity to receive a higher grade than the one they received individually. Haberyan and Barnett (2010) examined the efficacy of collaborative testing in an Educational Psychology class. Students completed the first and final exam individually and had the option of completing the third exam collaboratively. The researchers found that those testing with a partner scored significantly higher (M=9.63, SD 2.20) than those testing alone (M=8.15, SD=2.60). A second part of the study was used to determine effects of exam performance conditioned on whether students studied with a partner. Study authors found that exam performance was improved with collaboration, regardless of whether the student studied alone or with a partner. Sainsbury and Walker (2008) studied group testing with students in a Bachelor of Pharmacy program in which thirty-five percent of the course grade was based on quizzes. In the study, quizzes were administered individually, then to a group. After discussion with their peer partners, students were permitted to either keep their initial answers or change them. Immediately after the quiz was submitted, course faculty allowed students to continue to engage in discussion about the quiz to enhance learning of material. Giuliodori, Lugan and DiCarlo (2008), studied students in a Veterinary Physiology course who took individual tests then answered the same questions in teams of two. Students who tested in a group answered questions correctly 70.2% of the time while students who tested individually responded correctly 58.7% of the time [t = 11.4, df = 137, p < .001, 95% CI: 1.88-2.67]. In a study by Zimbardo, Butler and Wolfe (2003) students in a psychology course were given the option of taking their midterm or final examinations with a partner of their choice. The final grade obtained by the team was considered the common grade. The researchers found that students’ performance on the team exams was better than those who took individual exams yielding means of 73.1 versus 64.6 (t =7.25, df = 202.74, p < .0001). When repeated with students in a comparable course the following term, authors found nearly identical results. Students also reported that they experienced reduced test anxiety, that the collaboration gave them practice with negotiating differences with answers, and that the experience enhanced learning and comprehension of material. Several studies have demonstrated that collaborative testing helps to enhance all student test scores within a course. Wink (2004) studied collaborative testing for two exams in a Health Care Policy course in which students were placed in groups of six to ten. Students took the exam individually, and then again as a group. Scores were computed using an average of the individual and the group results, and demonstrated that student scores on both exams increased along with their final course grades. Researchers discuss grade inflation as a potential barrier to the use of collaborative testing (Wink, 2003; Wilder et al, 2007). To prevent grade inflation of poor performing students, different approaches have been reported by various researchers. Wilder et al. (2007) used an average of the individual and group grade to obtain the students final grade. Students who did not receive a passing grade on the individual exam were permitted to participate in the group test activity, but could not elevate their grade based on the group grade. Jensen et al., (2002), for a second quiz in an Anatomy and Physiology course, had students meet in their group where they were given one copy of the questions and one answer sheet to turn in. The researcher recommended a scoring rubric with points assigned for ranges of correct answers. As an example, a group grade of 8 to 10 out of 10 possible points resulted in 5 points added to their final test score. Their overall mean scores for students were all higher when compared to those taking individual quizzes (t = 20.3, df = 405, p < .0001). Overall, both studies found that cooperative testing produced better performance on exams than individualized exams, but did not greatly impact students’ final grades. Another barrier to the use of collaborative testing is the concern that performance on group exams does not translate to similar performance on comprehensive exams and retention of material. Woody, Woody and Bromley (2008) studied students who participated in an individual retest approximately three weeks after their initial group exam. The students formed groups of three or four in advance of the group examination. They found that students scored higher in a group format test (M=84.1%, SD=8.22) than in the individual format (M=75.5%, SD=12.56); however, collaborative testing did not result in later retention of course material. Leight et al., (2012) determined that the vast majority of students scored higher on group exams than on individual exams. In this study, students were semi-randomly divided into two groups based on their performances on the first exam. The second exam, which included class content from the first exam, was taken individually and then again in small groups of two to four. The research revealed that the positive effect of collaborative testing did not result in increased retention of content. Lastly, Sandahl (2010) randomly assigned students to groups of three or four, in which two exams were administered to a group, and two to individuals. Students in the group exam were not required to reach consensus on the answers. The researcher found that students who took the exams collaboratively scored significantly higher than those who took the exam individually (p < .05). However, there was no significant effect found on retention of material (Lamda (12, 262) = .921, p=.526). This review intends to determine whether collaborative testing in the classroom setting produces positive learner outcomes in the higher education setting. Not only will this review identify whether learner performance will improve students’ class grades when measured against their individual grades, but it will also assess student's perception of anxiety during test taking in either individual or collaborative format. The review will also evaluate the students’ evaluations of the collaborative process, their perceptions of confidence with test taking skills, and their overall perceptions of comprehension and retention of material. It is important that classroom teaching and learning continue to evolve. A greater emphasis has been placed on teamwork in the workplace. Collaborative learning may be one mechanism that can enhance the concept of teamwork. This review intends to contribute to the growing literature on collaborative testing by demonstrating its impact on teamwork. As the classroom expands and the learners change, new and creative formats of teaching and learning are also required. Although collaborative testing is a fairly new method of learning, this review will contribute to the growing body of literature and provide relevant information for both faculty practice and student learning. It is the aim of this review to assess the effect of collaborative, group, or double testing on learning outcomes for students in higher education settings. Specifically, this systematic review asks the following research question: What is the effect of collaborative testing on learning outcomes for higher education students? This review will look at the evidence on collaborative testing, defined as instances where two or more students work together to complete an evaluative exam. This review will only include adult higher education students, who are defined as students aged 18 years or older attending post-secondary institutions. We will not exclude studies on the basis of their participants’ socioeconomic status, gender, race or any other demographic variable. All comparative quantitative study designs with at least two time points for measurement of the outcomes—including but not limited to randomized control trials and quasi-experimental studies, pre-post and time series designs with control groups, as well as observational studies with control groups, such as longitudinal prospective or retrospective cohort studies and case control studies—will be included in this review. The use of these study designs in systematic reviews of comparative effectiveness is an acceptable strategy per the IOM (2011) and Campbell (2013) guidelines, particularly in the context of the GRADE (Guyatt, Oxman, Schünemann, Tugwell, & Knottnerus, 2011) approach for evaluating the quality of the evidence. Quantitative study designs that do not include a comparator/control group will be excluded, such as correlational designs, descriptive studies, or single cohort prospective or retrospective studies. Before-after studies in which participants serve as their own control will not be excluded from the analyses. The manner in which the baseline equivalence of the comparator group was established, or whether baseline equivalence was established at all, will not be a consideration in the selection of studies for inclusion. Strategy Development The search strategy will be developed in three stages. First, keywords derived from the study's inclusion criteria will be combined using Boolean operators and used to identify potentially relevant studies using the PsycINFO database (see Appendix I for this list). Citation chaining will also be used to identify additional relevant studies as well as those citing studies comprising the initial subset. Second, indexing of the identified citations will be examined for subject headings and relevant keywords capable of being added to terms of the search. Finally, the revised search strategy will be executed in the individual databases using controlled vocabulary including database-specific subject headings derived from database thesauruses and the revised list of keywords. The assistance of a librarian with experience supporting systematic reviews will be sought in searching published, peer-reviewed literature using validated search strategies for the following electronic databases: MEDLINE, Embase, Academic Search Premier, CINAHL, PsycINFO, ERIC and VET-Bib. In addition, the ProQuest Dissertation & Theses database and a general internet search engine (i.e., Google) will be used to identify literature not formally published in sources such as books or journal articles (i.e., grey literature) using a combination of keywords and outcomes of interest. Pubmed will also be searched for publications ahead of print. Databases will be searched from inception to the present date. Search results will be merged using appropriate systematic review software and duplicate records of the same report will be removed. Our search will not employ hand searching of individual We will to for which only are We will also within the with regarding other potentially relevant studies that they may be Finally, the search in the review will be to for to that the recent is and will the and of the studies that are identified from of the search and will determine whether these studies meet the inclusion criteria of this review. these will be first consensus among them with by of a third in The of studies that pass the initial by and will then be examined by two and in to that they meet the inclusion these two authors in this process will be first by or if in with a third The critical provide means of whether or not the study to with these inherent in observational designs, such as the use of as a strategy to for or testing on demographic to identify potentially significant differences the comparator groups. The presence of these strategies will be identified and examined as part of the of the studies that meet the inclusion We will use the GRADE et al., 2011) approach to or the evidence from studies or observational studies, We will not exclude any studies based on on individual of bias or study of of the two will be first through then if in with a third will be using a in which will be tested on of the included studies the two who will be performing the and will be consensus or in with a third The that will will include information Study study post-secondary of type of collaborative testing example, then group then group testing timing or group testing without an individual of the comparator type of of how the is measured type of instrument was used to measure the if information on the effect size for each of and for mean for size in each group, measure of in the study confidence or and studies will be for We will outcomes mean differences in mean grade in anxiety scores measured using validated as or mean differences to allow of studies using different outcomes such as grade inflation as the of students who pass the course who otherwise would not will be as or The of all outcomes will be reported using the 95% confidence We do not to students’ of the collaborative process, or perceptions of comprehension on used within individual studies. these outcomes are will results to a We will also studies methods of testing double testing, dyad testing, and group in the Finally, will be where and for that are not to student is will be to identify among multiple studies. evaluating the outcomes of using methods to be into will either be combined for or using the will not be in the that the majority of studies are to present a high of bias on study that would their will be in using the effects described by & We effects for multiple reasons. To the literature on collaborative testing a of effects on student that the intervention effect described by these studies is not identical but along a Second, (i.e., individual study of the effect to vary by effects for to be to study such as selection method of collaborative testing group subject that a more approach is (i.e., a effects Finally, in the of true the effects is reduced to the We will perform a based on the of bias criteria for each studies. each of bias will exclude studies that are found to be at high of a systematic and will the without these studies in to determine whether that type of of bias has an impact on the of the will be to for (see bias will be if there are at least 10 studies available to in the We will assess bias using in using best 2011). We will not formally assess bias using The will a to study authors in to We will whether the that is to be at or whether it is not at but is to bias and studies will be excluded from the on the basis of to the outcomes of interest. will be in which studies with are excluded to the effects on of effect will not be We will employ to the potential impact of the following which are a on the effect size of the mean of the gender, study collaborative testing group and methods of testing multiple choice, answers, We will also use to the impact of each type of systematic bias identified in the studies on the effect size for each studies. will be in will be examined individually as It has been that effects may contribute to collaborative testing performance as learner improve 2009). It that older learners may have had more opportunity to learning strategies than in higher education the impact of learner on effect may and (2007) reported that no and for collaborative testing individual students were more to perceive with peers as lower anxiety and to higher test then students on be more to receive the of the impact of on effect also be of questions may more to student collaboration than and consequently impact both learning and student performance. individual studies the impact of collaborative testing on only a single type of examination. would allow for of different on effect the same it to that course content may be an important and as such will also be adult and and or or Anxiety or or or or or or perception performance or grade or Anxiety or Test scores or average or or inflation or or to on this review will of the by the Group of the Campbell Collaboration and will be completed by