Leon Furze

AI chatbots like OpenAI’s ChatGPT have kicked the hornet’s nest in education. From articles declaring the “end of everything” (writing, the English classroom, the entire education system…) to claims that AI will revolutionise education, we’ve seen the full spectrum of hype.

What we’re yet to see much of, however, are practical solutions for dealing with these new technologies. I’m a PhD student, studying AI in education. I’m also a former secondary school teacher, Director of Learning and Teaching, and Head of English. As part of my consulting, I go into schools every week and hear about the challenges of AI and I write a lot about how AI is being adopted in schools. I turned one of my blog posts, Practical Strategies for ChatGPT in Education into a PD which has now run for over 800 educators in secondary and tertiary, online and face to face.

One area where AI is causing a significant stir is assessment. Amidst all of the end-of-everything narrative and fears about students becoming compulsive plagiarists, some schools and universities are trying to adopt sensible approaches to incorporating AI into their assessment tasks.

I’ve been paying close attention to how people are using AI like ChatGPT. This blog post introduces the AI Assessment Scale, a five-point scale that can help educators determine the level of AI involvement in their assessments.

The AI Assessment Scale: A Closer Look

The AI Assessment Scale ranges from “no AI” to “full AI” and encompasses different levels of AI integration. Here’s a breakdown of the five-point scale:

1. No AI: The assessment is completed under supervision, and/or handwritten, and/or under exam conditions. This level is suitable for testing knowledge and comprehension. For example, a traditional multiple-choice exam or an in-class essay written without the use of AI tools.
2. Brainstorming and ideas: AI can be used in the initial stages of the assessment for brainstorming and idea generation, checking ideas, etc. This level is suitable for assessments where students need to demonstrate their writing skills, such as constructing their own essays. For instance, students might use AI tools to help generate ideas for a persuasive essay or research paper.
3. Outlining and notes: AI can be used to outline entire responses or convert notes (or audio transcriptions) into organised ideas. This level is suitable for assessments where the focus is on the final quality of the writing, word choice, and expression. For example, students might use AI tools to turn their handwritten notes into a cohesive essay outline or restructure their notes to create a more logical flow of ideas.
4. Feedback and editing: AI can be used to provide feedback, self-assessment, or editing and revision. This level is ideal when the assessment focuses on the quality of the ideas and understanding, independent of the quality of language and expression. Students might use AI tools to receive instant feedback on their draft essays or identify areas of improvement in their writing.
5. Full AI: AI can be used to generate the entire output. This level is suitable when the outcome of the assessment is judged on the earlier organisation, idea generation, discussion, orals, and other methods. For example, students might use AI to create a comprehensive summary of a group discussion or synthesise research findings into a cohesive report.

Let’s take a look at each of those in more detail, with a few examples. A PDF of this information can be downloaded for free at the end of this article.

Scale Level	Description	Examples of Assessment Tasks
1. No AI	The assessment is completed under supervision, and/or handwritten, and/or under exam conditions.	1. Students complete a traditional multiple-choice exam on historical events. 2. Students write an in-class essay about the impact of technology on society without the use of AI tools. 3. Students solve a series of maths problems on paper during a timed examination.
2. Brainstorming and ideas	AI can be used in the initial stages of the assessment for brainstorming and idea generation.	1. Students use AI to generate ideas for a persuasive essay on the advantages and disadvantages of social media. 2. Students use AI tools to brainstorm potential solutions to an environmental problem in a group project. 3. Students collaborate with AI to develop innovative business ideas for a mock start-up pitch competition.
3. Outlining and notes	AI can be used to outline entire responses or convert notes into organised ideas.	1. Students use AI tools to create an essay outline on the factors contributing to climate change based on their research notes. 2. Students use AI to convert their handwritten notes on a novel into a structured analytical essay outline. 3. Students use AI to organise their research findings on public health policies into a clear presentation outline.
4. Feedback and editing	AI can be used to provide feedback, self-assessment, or editing and revision.	1. Students submit their draft essays on the ethical implications of genetic engineering to AI for feedback on structure, clarity, and persuasiveness. 2. Students use AI tools to receive instant feedback on their oral presentations and improve their delivery. 3. Students collaborate with AI to revise and edit their group research papers on the effects of globalisation on local economies.
5. Full AI	AI can be used to generate the entire output.	1. Students provide AI with their research and ideas, then use the AI-generated synthesis to create a comprehensive report on the future of renewable energy. 2. Students input their group discussion notes on the challenges of urban planning into AI to generate a cohesive summary. 3. Students supply AI with their design concepts and requirements to generate a visual representation of a proposed architectural project.

Benefits and Challenges of Incorporating AI into Assessments

Incorporating AI into assessments could several potential benefits, such as enhancing the creativity, writing quality, and efficiency of feedback processes and self-assessment. However, there are also challenges to consider. Ensuring that AI does not replace students’ critical thinking and problem-solving skills is crucial, and something which I know from experience many parents are concerned about. The ethical concerns around AI-generated content and balancing the use of AI with traditional assessment methods must be addressed.

Applying the AI Assessment Scale to Non-Writing Tasks

The AI Assessment Scale could easily be applied to other types of assessments, such as oral presentations, group projects, and problem-solving tasks. Furthermore, it can be applied to other AI applications like image generation, data analysis, and virtual simulations. In each case, educators can use the five-point scale to determine the appropriate level of AI integration.

Think about how this Scale could be applied to a visual arts assessment, for example. Students might use no AI in the initial idea generation, and then supplement their brainstorming with an image generation app to look for inspiration – much in the same way students might already use apps like Pinterest. They could create digital artwork themselves, with the use of AI, or using AI to entirely generate the artwork and then editing it themselves.

Creating Clear Guidelines for AI Use in Assessments

The idea behind the AI Assessment Scale isn’t to “catch” students in the act of cheating. In fact, it’s getting increasingly difficult to even define what we mean by “cheating”. The Scale should be used as a discussion point, and maybe as an addition to a task sheet to clearly indicate how students are permitted or encouraged to use AI for a given task.

It might also be appropriate to break a task down into different elements, and apply different levels of the Scale to different parts of the assessment. For example:

• Your initial brainstorming must be done by hand, on paper, using your knowledge and ideas only. (Level 1)
• Your ideas can then be refined and developed using AI and a thinking routine like Generate-Sort-Connect-Elaborate. (Level 2)
• Once you have developed your initial ideas, your first draft outline should be completed by hand in class. (Level 1)
• You may use AI to expand on your outline and suggest improvements to clarity, logic, and the overall structure. (Levels 3 and 4)
• Once you have completed your draft writing, you may use AI to proofread and make recommendations. (Level 4)

Using ChatGPT with the Assessment Scale

Finally, here are a few examples of how a student might approach an assessment task using prompts in ChatGPT.

No AI

Sorry, no prompts here!

Brainstorming and ideas

Generate a list of features that a teenager might be looking for in a smartphone app that helps them track their environmental waste.

Here are three of my ideas about <topic>: <copy/paste ideas>. Add onto each of those ideas with three connected ideas each.

Outlining and notes

These are my notes from a lesson on <topic>: <copy/paste notes>. Turn them into an outline for an essay on the question <question>

This is a transcript of an audio brain-dump. Turn the transcript into organised notes with headings and subheadings: <copy/paste transcript>

Feedback and editing

Role play: You are my secondary school Year 12 English teacher. You haven’t had your morning coffee. Provide some brutally honest feedback about the quality of my arguments: <copy/paste draft>

Check the grammar, spelling, and punctuation of this and make recommendations: <copy/paste draft>

Full AI

This is the audio transcript of a group discussion on <topic>. Use it to generate the outline for a PowerPoint, and the outline for a report.

Use the outline for the report you created to write the full report. Begin with the first section and wait for our feedback.

If you found this post useful, please share it with your colleagues and consider joining my mailing list for future articles:

Download a free PDF copy of the scale:

I am working with a variety of schools on professional development and the creation of AI policies, including communications with parents and the community. If you would like to talk about AI in your school please use the contact form below: