Dr Leon Furze

This post is part of a series on developing generative AI strategy from faculty leaders and teaching and learning teams. Having been both a faculty leader and a Director of Teaching and Learning, I understand that a great many school strategies get lost in translation when they reach middle leadership and have to be put into action.

Because artificial intelligence will impact every discipline, I think we need to lead some of this change from the middle. If you haven’t already, check out the previous post in this series, first, outlining the whole strategy, then attacking your assessments and carrying out some small experiments.

Assuming you’ve followed the steps and attacked your assessments, it’s now time to look at the assessments you already have and find ways to update them to account for this technology. I don’t want faculty leaders and educators to completely redesign all of their curricula and assessments. Some tasks will need significant revisions and others only small tweaks.

If you are not familiar with the Artificial Intelligence Assessment Scale, developed by Dr. Mike Perkins, Dr. Jasper Roe, Associate Professor Jason MacVaugh, and myself, then I recommend you familiarise yourself with the following resources before going any further with this article:

• Assessment Scale Version 2 Blog Post
• AI Assessment Scale Pilot Project Blog Post
• AI Assessment Scale eBook
• JUTLP Volume 21 Number 6: AI Assessment Scale Pilot
• The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment (preprint)

You can access a free ebook on the AIAS with over 50 activities for the 5 levels by signing up for the mailing list here:

This comprehensive set of resources explains the rationale for the AI Assessment Scale and why such a scale is necessary. It includes lots of examples across disciplines which apply to both K-12 and higher education.

The AI Assessment Scale in Brief

We developed the AI Assessment Scale because we acknowledged that schools and universities needed something more nuanced than a binary “use/don’t use” approach to this technology.

Students have also told us through various studies that they want education providers to help them understand how to use generative AI technologies responsibly and ethically. Here in Australia, the national Framework for Generative AI in Schools includes two explicit guiding statements which speak to the need for a structured approach to using artificial intelligence:

In Core Principle 1: Teaching and Learning, we find the following:

• 1.5 Learning design: work designed for students, including assessments, clearly outlines how generative AI tools should or should not be used and allows for a clear and unbiased evaluation of student ability.
• 1.6 Academic integrity: students are supported to use generative AI tools ethically in their schoolwork, including by ensuring appropriate attribution.

These tools are not to be banned, and students must be supported in their use, but assessment tasks need to be absolutely clear on how AI can and cannot be used in a given situation.

The AI Assessment Scale runs from Level 1 to Level 5, ranging from No AI to Full AI. Our most recent version, published in the Journal of University Teaching and Learning Practice, is our most up-to-date version.

Auditing Your Assessments

Rather than starting from scratch with all of your assessment tasks, or panicking that generative AI means you’ll have to throw everything out, I suggest you carry out an audit of your major summative tasks to see if they already align with one of the five levels. To help with this process, we’ll be developing an AI assessment audit tool which you can use in a faculty meeting or distribute to your team members and have them audit the assessment tasks that they are responsible for.

Until that tool is available, you can carry out the process yourself by placing existing assessments at the most appropriate level and discussing how the task might be adjusted to bring it up or down. Avoid getting bogged down – limit this to the really crucial graded assessments and end-of-unit assessments. Or, if you have project-based learning, align them with the overall grade and final criteria for the whole project.

It may be useful to have some double-up if you have multiple faculty teaching the same assessments. And then do this individually rather than as a group for a diverse range of perspectives – the more ideas you can collect, the more robust your final judgment will be. Here are some questions you might ask:

• How much does it matter if the student remembers or memorises content?
• Is it an examination? Does it need to be?
• How much does it matter if the student is demonstrating their ability to write or their literacy skills?
• How much does it matter if the task involves creativity or critical thinking?

None of this is set in stone. You may find cases where your assessments don’t neatly fit into one particular area of the scale. We designed the scale to be flexible from day one and published it open access so that you can take the idea of the scale and adapt it to your own context.

After auditing your assessment tasks, you may decide to create a version of the scale which is more suitable for your organisation or faculty. For consistency’s sake, I would recommend having an institutional version of the scale which allows some variations in the wording for different faculties but keeps the integrity of the levels.

After the Audit

After staff have completed the audit, I would recommend the faculty leader collates this information and then presents something back during a meeting. You may find some inconsistencies between individual staff members’ perceptions of assessment tasks. You might find that certain staff wish to do everything at the “No AI” level because of concerns that students will cheat or misuse the technology. These are valid concerns, and you as the faculty leader needs to manage any of those conflicts.

In the spirit of the previous step, “Bullets then Cannonballs,” I would suggest that you pilot the AI Assessment Scale only for the assessments in one term, or focus on a particular subject or year level before attempting to apply the AI Assessment Scale to all of your assessments.

In the next step, we will look at evaluating and communicating what you have learned with colleagues and leadership.

I regularly work with schools, universities, and faculty teams on developing guidelines and approaches for Generative AI. If you’re interested in talking about consulting and PD, get in touch via the form below: