Selecting Effective Examples to Train Students for Peer Review of Open‐Ended Problem Solutions

Background Students conducting peer review on open‐ended problem solutions require training. For that training, the selection of training examples is critical. Purpose This study explored how the characteristics of five example solutions used in training and their associated expert evaluations affected students' abilities to score peer team solutions on a model‐eliciting activity (MEA). Design/Method For this training, individual students reviewed a randomly selected example solution to an MEA. They were then asked to compare their review and an expert's review. Students were then assigned a peer team solution to review. An expert later rated the five training examples and 147 team solutions that had been peer reviewed. Differences between the scores assigned by the expert and a student to a training example and a peer team solution were used to compute the student's improvement as reviewer from training to peer review. ANOVA testing with Tukey's post hoc analysis identified differences in improvement based on the training example students saw during training. Results Statistically significant differences were found in the number of errors students made during peer review depending on the quality of the example they reviewed in training. Specifically, a low‐quality example and associated expert evaluation resulted in more accurate scoring during peer review. Conclusions While students typically ask to see exemplar solutions, this research suggests that there is likely greater value, for the purpose of training for peer reviewing, in having students see expert evaluations of lower‐quality examples.