Saturday, April 12, 2008

Electronic Peer Review

Kelly Whitney

Peer review is quickly becoming a fundamental practice in composition courses. Instructors rely on students to provide feedback for others with the hopes of improving both the writer and reviewer’s papers. As computers and technology become more prominent in the composition classroom, instructors are able to incorporate technology into fundamental activities such as peer review. More and more instructors are turning to online peer review for several reasons; some teachers find it to be more convenient for the students, some use it to help create a paperless classroom, others prefer to dedicate class time to instruction, and many find the feedback to be of higher quality than in traditional peer review. Regardless of the reason, instructors who do or intend to implement online peer review must be aware of several concerns: Pedagogical approaches to online peer review, how to train students to engage in online peer review, and the benefits of online peer review versus traditional peer review are all concerns that instructors must be aware of, and all of which are investigated in this study. This project draws from scholarly and timely sources that can be found in published journals and books from several disciplines.

Althauser, Robert, and Kim Darnall. “Enhancing Critical Reading and Writing Through
Peer Reviews: An Exploration of Assisted Performance.” Teaching Sociology 49
(2001): 23-35. JSTOR. Roesch Lib., U of Dayton, Dayton, OH. 02 March 2008. Althauser and Darnall, two researchers at Indiana University, produced a study to determine the correlation between the quality of peer review and the quality of revisions. The participants in the study were required to revise their papers twice, and in the second revision they were to use only peer comments (no instructor comments were given). The students were given specific criteria for peer review; they had to comment on specific aspects of the text (clarity, coherence, syntax, etc.). The results of this study suggest that critical thinking skills are developed through the revision process as improvement was seen in each revision; higher quality peer reviews yielded higher quality revisions. The researchers assert that revision is the key to improving student writing and that peer review is essential to writing development. This article provides useful insight into the benefits of peer review; however, there is no analysis regarding the role that online peer review played. Thus, this source will not be useful for my project.

DiGiovanni, Elaine, and Girija Nagaswami. “Online peer review: an alternative to face-to-
face?” ELT Journal 55.3 (2001): 71-85. Academic Search Complete. Roesch Lib.,
U of Dayton, Dayton, OH. 02 March 2008.
This article evaluates whether instructors should conduct peer review traditionally (face-to-face) or electronically. The researchers trained their students how to perform both face-to-face peer review and online peer review, and all peer review sessions were conducted in class. All students participated in both approaches to peer review, and almost all found peer review to be useful. Most students were indifferent to the approach that was used, possibly because both methods were used in class. Online peer review is generally considered to be beneficial because it is conducted outside the class, on the student’s own time, and in the comfort of her own home, all of which these students were unable to experience. The researchers found that students stayed on task and remained focused longer with online peer review as opposed to face-to-face peer review. The researchers encourage instructors to use the online approach to peer review for several reasons: Online peer review allows students to consider and reflect on their thoughts before writing them, to retain suggestions and information better, and to foster computer literacy. The pedagogical approach to training students for online peer review and the benefits that result from it are relevant to my research.

Lin, Sunny San-Ju, Eric Zhi-Feng Liu, and Shyan-Ming Yuan. “Web Based Peer
Assessment: Attitude and Achievement.” IEEE 44.2 (2001): 13-27.
This study identifies a positive correlation between students’ attitudes toward online peer review, quality of feedback, and writing performance. The researchers used online peer review during several steps of the writing process, including brainstorming, drafting, and revising. The participants of this study engaged in identifiable online peer review with multiple reviewers. In a post-test survey, students overwhelmingly revealed that they preferred online peer review to face-to-face peer review. The reviewers suggest that allowing students to discuss their writing assignments online will foster more positive feelings toward online peer review. Furthermore, they emphasize that instructors must be aware of their students’ attitudes; teachers need to actively care for students’ feelings and attempt to create more positive feelings toward online peer review in order to achieve the benefits that online peer review provides. Students who do not hold positive attitudes towards online peer review will not perform as well, and their writing performances will suffer as a result. This article illustrates the importance for students to maintain positive attitudes, and suggests pedagogical strategies to ensure student interest, both of which are significant for my research.

Lu, Ruiling, and Linda Bol. “A Comparison of Anonymous Versus Identifiable e-Peer
Review on College Student Writing Performance and the Extent of Critical
Feedback.” Journal of Interactive Online Learning 6.2 (2007): 100-115. Education
Research Complete.
Roesch Lib., U of Dayton, Dayton, OH. 02 March 2008.
Using multiple reviewers in an online peer review format, this study compares the quality of feedback from anonymous reviewers and identifiable reviewers. The participants were split up into two groups: the test group consisted of multiple anonymous reviewers in an online format, and the control group consisted of multiple identifiable reviewers in an online format. The results concluded that the anonymous group provided more critical feedback than the control group; the feedback was more honest and critical because students didn’t have to worry about hurting others’ feelings. The researchers assert that this approach to peer review is most effective because of deindividuation; group members don’t think of others as individuals, and they also feel they can’t be singled out by others in the group. Thus, anonymity is necessary for deindividuation to be successful. Because the students don’t have to worry about judging or offending another student with anonymous peer review, more critical feedback is given. This article will help establish the most effective way to conduct peer review, which is a crucial aspect of my research.

MacDonald, Janet. “Exploiting Online Interactivity to Enhance Assignment Development
And Feedback in Distance Education.” Open Learning 16.2 (2001): 179-189.
Academic Search Complete. Roesch Lib., U of Dayton, Dayton, OH. 02 March 2008.
MacDonald identifies two concerns in this article. First, she argues that students must learn how to critically assess their work in order to improve their writing. Thus, she requires her students to conduct online peer review to help develop self-judgment; as students evaluate others’ works, they can examine their own work more critically. Furthermore, online peer review allows students to see alternative approaches to writing, different styles of writing, and gives them more practice learning about writing within their genres. Second, MacDonald identifies students’ inabilities to adjust to reading and writing in new courses. By engaging in online peer review, the students move to a greater involvement in assessment, which forces them to analyze and reflect more critically on their own work. They are then better able to rhetorically analyze their new writing situations and respond accordingly to their new writing courses. The students in this study, however, admitted that they struggled with how to approach online peer review; they felt that they could have gained more from it if they had a better understanding of how to approach online peer review. Thus, MacDonald claims that teachers must train or guide their students through online peer review in order for it to be successful. The information this article provides regarding both the benefits of online peer review as well as the drawback when not executed properly are significant for my research.

Robinson, Jennifer. “Computer-assisted peer review.” Computer-Assisted Assessment in
Higher Education Ed. Sally Brown. London: Kogan, 1999. 95-102.
Robinson acknowledges the numerous institutions that use multiple reviewers and asserts that student writing should also employ multiple reviewers in peer assessment. She identifies the teacher’s role to that of an editor, and the students’ roles to those of reviewers. Robinson conducted a study to determine whether anonymous or identifiable peer review is more effective. The first case was a double-blind study, where both reviewers and writers remained anonymous. The second case identified the writers, but the reviewers remained anonymous. The students overwhelmingly preferred anonymous peer review, claiming that there is no need to know who the writer or reviewers are. Furthermore, reviewers in the second case felt uncomfortable reviewing papers of students whom they knew. In both cases, however, reviewers tended to give numerous grammatical feedback, indicating they were aware of what aspects of the paper they should have been reviewing. Robinson encourages teachers to instruct or train their students on how to perform electronic peer review, but doesn’t offer any suggestions for this training. Robinson continues to explain the benefits of online peer review over face-to-face peer review: Online peer review allows for many more readers, much more feedback, and when students are trained properly, higher quality and better focused feedback. Although most of this article discusses programs and potential software for online peer review, Robinson provides more insight into the benefits of peer review which is pertinent to my research.

Topping, Keith. “Peer Assessment between Students in Colleges and Universities.”
Review of Educational Research
68.3 (1998): 249-276. JSTOR. Roesch Lib., Uof Dayton, Dayton, OH. 02 March 2008.
This article is a synthesis of all the studies and research conducted regarding peer review. Topping identifies and explains specific areas of peer review, including peer assessment of professional skills, group work and projects, and oral presentation skills. He dedicates one small section to the research of computer-assisted peer assessment. He finds that this approach to peer review is becoming more popular, but there is still comparatively little data regarding its effectiveness. What research is continuing to support, however, is that writing performance is increasing and more words are produced in online peer review. What’s most interesting about this article are his seventeen parameters for peer review typology. He creates a table that examines the requirements and rhetorical situation of the peer review to ensure students have the best opportunity to benefit from the peer review. Although this article offers superior information regarding the peer review movement to date, it will most likely not be used in my project. This article would better suit an educator who wishes to become aware of the numerous aspects and approaches to peer review instead of concentrating on online peer review.

Xu, Yi. “Re-Examining the Effects and Affects of Electronic Peer Reviews in a First-Year
Composition Class.” Reading Matrix: An International Online Journal 7.2 (2007):
1-21. Education Research Complete. Roesch Lib., U of Dayton, Dayton, OH. 02 March 2008.
This study investigates the quality and quantity of comments in traditional and online peer review. Xu examined nine students’ responses on four drafts, which is an incredibly small research sample. Two of the drafts were reviewed online and two were reviewed traditionally. The results indicate that there is no difference between the language styles and comment content between the two modes. Furthermore, the quality and quantity of reviewers’ comments remained the same between the two modes. These results are much different than what other researchers have yielded, and the research variables could be why these outcomes are much more different: The drafts that were reviewed traditionally contained about eight hundred more words than the drafts that were reviewed online, which creates more opportunities for comments. Furthermore, the number of comments decreased with each draft. Xu suspects a loss of interest in peer review is to blame for this; however, their claims could have been validated had they interviewed or surveyed the students regarding their feelings towards the reviews. Xu encourages teachers to use innovative approaches to keep peer review fresh and to maintain student interest. Because this study was poorly executed and its results are questionable, this will not serve as a useful source for my research. There are several other studies—studies that were conducted more thoroughly—that argue online peer review does, indeed, invite more critical comments.

No comments: