Key points are not available for this paper at this time.
Abstract Motivation: Development and evaluation of inclusive teaching practices is a vibrant research area. For four years, we worked to implement three inclusive practices: standards-based grading with reflection, co-created assessment rubrics, and peer review of written deliverables in a sophomore-level experimental design laboratory and lecture course. This study focuses on peer review effectiveness motivated by the work of Saterbak, et al., which showed a strong positive correlation between instructor review and peer review in a biomedical engineering laboratory and suggested that peer review could provide effective feedback. Further, peer review resulted in the perceived improvement of the students' ability to critique. We hypothesize that our implementation of peer review leads to increased achievement over time and positive student attitude. More specifically, our research questions are: 1. Do students provide high quality feedback? 2. Is overall class performance improved as assessed by course standards? 3. Is there a favorable student attitude towards the peer review process from the viewpoint of the critic and critiqued? Methods: Students completed draft and final two-page abstracts individually and provided reciprocal peer feedback in pairs. A research assistant scored the draft and final versions of the extended abstracts using the course's student/instructor co-created rubric based on problem solving standards. These scores were detached from grades. Separately, all student work was graded by teaching assistants who underwent grader-calibration. The research assistant assessed peer review quality on a minimal grading scale (0-2 points) for each of the following: specificity, suggestions, justification, and appropriateness as previously described. Student attitude was assessed using our previously reported survey from the perspective of both the critic and the critiqued. Then, the following analyses were conducted: - Assessment of peer review quality over the term - Assessment of draft and final scores over the term with one-way ANOVA - Investigation of achievement in weak categories using one-way ANOVA - Investigation of student attitude regarding peer review using 25 Likert-scaled questions where "4" refers to "strongly agree" and "1" refers to "strongly disagree" Results: Peer Review Quality: In terms of peer review quality analysis, students excelled at commenting "appropriately"; however, there is room for improvement with respect to "specificity", "justification", and the ability to provide "suggestions". The overall peer review quality was 35+/-9 out of 48 points. Improvement in Achievement over Time: We saw a significant improvement in achievement between the Abstract #2 draft and the Abstract #3 draft (p=0.046). There was a trend for improvement between the Abstract #1 draft and the Abstract #2 draft (p=0.07, n=16-17). No statistically significant differences existed at the final abstract stage in terms of achievement over time. Additionally, we did not find statistically significant improvements in the two standards with which students typically struggle – "problem identification" and "interpretation". Student Attitude regarding Peer Review: From the perspective of the critic, students rated "enjoying giving feedback" (2.59/4) and "feeling reluctant to give negative non-anonymous feedback" (2.59/4) the lowest. Conversely, students reportedly understood how to assess extended abstracts (3.55/4) and had the confidence to assess others' extended abstracts (3.55/4). From the perspective of the critiqued, students provided the highest rating to "believing it is important for students to learn how to implement the feedback that is provided to them by peers" (3.50/4). The lowest ratings corresponded to "it is tough to write an extended abstract knowing it would be evaluated by another student without it being anonymous" (1.86/4). Discussion and Conclusion: Students may struggle with "specificity", "justification", and "providing suggestions" during peer review as they may not have sufficient time and understanding of their peer's work to provide higher-level comments. Students may benefit from additional instruction of peer review. It is possible we only observe an improvement in achievement in draft scores over time, but not final scores, because students were not willing to dedicate as much time for draft preparation as they did for the final versions (which all rose to a high standard throughout the quarter). Draft scores potentially improved due to increase in skill, efficiency, and familiarity with abstract format. We may not see improvements in "weak" areas because the students were not able to provide high quality review in these areas. The fact that students favorably viewed non-anonymous peer review could be because of the strong classroom community. Further, pair-style peer review may be a best practice for implementation. In summary, peer review is an inclusive teaching practice that has the potential to improve draft quality, may require additional instruction, and is viewed favorably by students.
Ankeny et al. (Sun,) studied this question.