Guidance on Moderating: Scenarios
The guidance below was circulated to all TEIs in June 2016. The guidance is not a change of policy, but rather additional information to help TEIs interpret and apply University policy in the context of the Common Awards. The guidance will be discussed at the Common Awards staff conference in July 2016. It will be revised in light of feedback received from TEIs. In the meantime, it is hoped that the draft guidance will be of use to staff in TEIs.
The University has published revised guidance on double marking and on the moderation of marks. The revisions clarify the existing policy on the moderation of marks, in particular restating the insistence that
moderation ‘is not about making changes to an individual student’s marks’; moderation ‘seeks exclusively to identify systematic defects in the first-marking process’ (§19).
Some questions have arisen around the TEIs as to how this is supposed to work, and what moderators should do in cases where they do disagree with individual marks. The following scenarios are intended to clarify the ways in which moderation can work.
Anne, acting as moderator, looks at several of the essays in the sample given to her, in the light of the assessment criteria, and decides that she would have given a different mark to them. She recognizes, however, that Bob’s first marks are fully justifiable. So Anne does nothing.
The point of moderation is to check, and to provide feedback on, the first marker’s marking of the work. It is not about balancing two people’s independent judgment in order to arrive at a final mark; it is about checking whether the first mark is justifiable.
Charles, moderating a sample of a set of ‘resources for others, with commentary’, decides that the marks given by Danielle are in general too high (though rather unevenly so). So he gets in touch with Danielle, and discusses it with her, and they agree that she has tended to underplay the need for the commentary to be critical, not merely descriptive. Having discussed those reasons with Charles, Danielle goes back to the whole run of pieces of work, and adjusts her marks and comments, checking in with Charles to make sure that she’s now doing the right kind of thing.
Where moderation leads to suggested changes, those changes should be applied to the whole run, not just the sample.
Esther, looking at the portfolios in the sample sent to her, decides that one of them has been given a lower mark than is justifiable. She explains this to the first marker, Francis, pointing out that in his comments he’s given a description (which Esther thinks is fair) that would suggest a mark somewhere in the low 80s according to our marking criteria – yet he has only given a mark of 74. Francis and Esther discuss the matter, and Francis agrees that he can be too reluctant to give high first class marks. So Francis looks at the other first class marks he has given (and only those), and rethinks the marks he has given to all of them, dropping Esther an email to let her know what he was done and make sure she’s happy with it.
When deciding which pieces of work need to be revisited in the light of moderation, markers should use their common sense to identify the other work to which the moderator’s feedback might be relevant.
Graham, looking at one exegesis in the sample he’s moderating, thinks that Helen’s mark is unjustifiably low. He and Helen discuss the case, and Graham explains how he thinks the marking criteria demand a higher mark. Even after that discussion, however, Helen remains convinced that the essay should be given a lower mark than Graham thinks is justifiable. At this point, the case is handed over to a third marker, Imogen. Imogen looks at the anomalous essay, but also other essays from the run, in order to get a proper sense of why this one mark is causing dispute, and she looks at both Graham and Helen’s comments. She could simply have looked at the same sample of essays that Graham considered, but she decided that the nature of this dispute made it sensible for her to consider a rather different sample. Finally, Imogen, in discussion with Graham and Helen, suggests a resolution.
The policy allows for recourse to a third marker, in cases where there are anomalies.
Kerry is first marker for a set of reflective journals. She is genuinely unsure whether two of the journals deserve a mark at the top of the 60s, or just over the boundary into the 70s. She sends those two pieces of work to the moderator, Luke, alongside the sample she sends for moderation, and she specifically asks him to advise her on whether to give first class marks. Those two essays are not part of the sample for moderation, and this request for advice is separate from the moderation process.
Moderation itself should not lead to the changing of individual marks without revisiting all the relevant work, but that does not mean that markers cannot also consult one another in other ways.