Leon Furze

If you’ve ever formally assessed senior English (at least here in Victoria), then you’ve used comparative judgement to assess writing. Comparative judgement, as the name suggests, is an assessment method based on comparing responses to determine their overall placement on an assessment scale. It’s an effective method supported by research, much of which is outlined in Emina McClean’s literature review published earlier this year for AERO.

In this post, I’m going to explore the pros and cons of comparative judgement and outline a method for teachers to work together to improve the assessment of writing in any curriculum area.

Setting the expectations

As with any kind of written outcome, it’s important to set the expectations before the assessment. Comparative marking is generally a summative assessment method used with a final written outcome, but other formative methods of assessment and feedback can and should be used along the way.

Establish clear expectations for what students are going to have to write by developing criteria or expected qualities. The best way to do this is to sit with other teachers working on the same unit, and ask some of the following questions:

• Is there an “ideal” form for the outcome of this task?
• Is it worth presenting multiple options, or letting students choose from any form?
• What (knowledge/skill) is being assessed, and what form(s) are the most suitable vehicle for those outcomes?
• Is a written outcome the best decision for what we want to assess? (As opposed to, for example, an oral, a discussion, a visual or multimodal text)
• What common elements do we wish to assess that can be separated from the form?

Once you have identified the most appropriate form of assessment, you can work on the overall expected qualities. The VCE English Examination provides a simple example of this, with each section of the exam scored on a 0-10 scale. Note that there is no large rubric attached to this: one advantage of comparative judgement is that rather than poring over a table of descriptors, the final grade is agreed upon by benchmarking and comparison.

A process for comparative marking

Comparative marking for writing tasks is effective but takes a while to establish. It requires benchmarking and moderation processes, and an acceptance of the fact that these are not finite judgements: a ‘7’ one year might be more like a 6 or even a 5 the next, dependent on the cohort.

The advantages of this marking process is that you quickly build a collection of benchmarked examples of student writing which can be used year-after-year. These benchmarks become an invaluable resource for future students and improve the overall quality of writing over time as it makes your expectations much clearer.

Here’s the process:

1. Ensure every teacher assessing the outcome is familiar with the expected qualities.
2. Collect student work for assessment. Each teacher roughly sorts their assessment pile into high, medium, and low.
3. Each teacher selects one high, one medium, and one low script to submit for benchmarking. For example, 4 teachers of Year 10 would submit a total of 16 papers to benchmark.
4. Either separately or as a group, each benchmark paper is awarded marks based on the expected qualities. Note that the qualities are not specific to a particular form, so it is possible to assess a blog versus a short story, or an essay versus a piece of flash fiction, and so on.
5. Discuss the benchmarks and agree on the final marks. These 16 papers now become the basis for the rest of the comparative marking.
6. Each teacher marks their individual pile by comparing their pieces to the benchmarks.
7. A final moderation meeting – which can be virtual and asynchronous – is held to allow teachers to double check, clarify, and question grades.

This style of marking relies more on a wholistic approach to the piece being assessed than a criteria-based rubric with levelled descriptors. For example, for a student to get a ‘7’ they need to have met the expected qualities on balance. We don’t penalise students for occasional mistakes, and we don’t negatively mark – we are looking to award what’s there.

Practical Writing Strategies – coauthored with Ben White – will include assessment advice for writing and activities that can be used to provide formative, ongoing feedback. For more information and for free resources and teaching ideas, join the mailing list.