Before you read on: We have updated the AI Assessment Scale and are now focusing our attention on Version 2. However, we understand that many schools and universities around the world have done great things with the original AI Assessment Scale and we continue to support V1. Make sure to check out both versions to decide which is best for you!
Back in April 2023 I published the first blog post about the ‘AI Assessment Scale‘ which ranged from No AI to Full AI, and was based on my experiences as an English teacher, and a student/educator grappling with the implications of Generative AI.
When Assoc. Prof. Mike Perkins, Dr Jasper Roe, and Assoc. Prof. Jason MacVaugh picked up the scale later in the year and we adapted it to this version, later published in Journal of University Teaching and Learning Practice (JUTLP), the AIAS began to take on a life of its own.
We’ve also seen many adaptations of the AIAS in both K-12 and Higher Education, reflecting the need for flexible, contextual frameworks that help educators and students to understand how AI can be used appropriately in their work.
AI and Academic Integrity
Since the release of ChatGPT in November 2022 the predominant narrative around AI in education has been around cheating and academic integrity. That has led to a proliferation of “detection tools” and other methods of trying to assure assessment security. I’ve written about the concerns extensively on this blog, since they’re such an obvious point of tension for K-12 and Higher Education providers.
- Generative AI, plagiarism, and “cheating”
- AI Detection in Education is a Dead End
- Teaching Writing in the Age of AI: Assessment and “Cheating”
- Beyond Cheating: Why the ban and block narrative hides the real threats of ChatGPT in education
- Ditch the Detectors: Six Ways to Rethink Assessment for Generative Artificial Intelligence
In the JUTLP paper on the AI Assessment Scale, we also address the concerns of Generative AI and academic integrity:
Prior to the public release of GenAI tools, the early 2020s had already seen education stakeholders place a renewed focus on academic misconduct and dishonesty, partly because of the COVID-19 pandemic, which led to perceived increases in cheating on behalf of students and teachers (Roe et al., 2023; Walsh et al., 2021). Simultaneously, an ‘arms race’ (Cole & Kiss, 2000; Roe & Perkins, 2022) between technology-enabled academic misconduct and detection software (for example, automated paraphrasing tools) was already in full swing. In this broader context, the focus on academic integrity violations in the era of the GenAI tools can be seen as one node in a network of existing conversations regarding the accelerating pace of digitalisation in HE and the resultant likelihood of what Dawson calls ‘e-cheating’, i.e. cheating that uses or is enabled by technology (Dawson, 2020, p. 4).
The Artificial Intelligence Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment https://open-publishing.org/journals/index.php/jutlp/article/view/810
We designed the AIAS partially in response to these concerns: if we can collectively find a way to distinguish between “appropriate” and “inappropriate” AI use, then we can find ways to move beyond the simplistic (and impossible to enforce) use/don’t use or “ban and block” approaches to Generative AI.
One major advantage of the AIAS was that it offered early “permission” for educators to experiment with AI, while still providing some boundaries for academic integrity conversations with students; and it is meant as a conversation starter – not a means of policing or “catching” students in the act of cheating with AI.
The AIAS encourages transparency and honesty regarding AI use, facilitating open conversations between educators and students. This transparent approach has been recognised by the Tertiary Education Quality and Standards Agency (TEQSA) in their advice to universities, The evolving risk to academic integrity posed by generative artificial intelligence: Options for immediate action, which includes the AIAS as one of three examples alongside University of Sydney’s “Two Lanes” approach, and UNSW’s adaptation which merges the two lanes with the AIAS.
There are several prominent examples of frameworks that help to make clear to students and staff alike what is appropriate and inappropriate use of AI in learning and assessment tasks. In many instances, a unit/subject coordinator will be best placed to decide on what is appropriate or inappropriate use of AI in a task. The following frameworks will assist with providing this clarity. If there is to be any enforcement of limits on AI use, the limits must be made explicit. Whatever framework each institution implements, what is particularly important is that clear guidance is available for everyone concerned. It is also critical to delineate what is allowable in learning and what is appropriate for assessed tasks where it needs to be clear what work a student has done themselves.
But can it stop students from “cheating”?
Despite our hopes that the AIAS can make use of the technology more transparent, we’re also pragmatic about the risks of generative AI for assessment security and “cheating”. As the TEQSA advice states, there is no clear data on how many students are using AI, but “estimates range from approximately 10% to over 60% of cohorts, with an unknown proportion of this use being inappropriate.”
Realistically, the AIAS – or any other framework, technological solution, or approach to academic integrity – cannot stop students from using Generative AI in ways which might be considered dishonest. Our Level 2, for example, suggests students can use AI for initial note taking, ideas, and organisation, but then the final work must be their own: but how do we guarantee that is the case, especially in light of increasingly capable models?
One response to the problem comes from a paper published today by Phillip Dawson, Margaret Bearman, Mollie Dollinger, and David Boud, which takes a different approach to “cheating” entirely.
In Validity matters more than cheating the authors argue convincingly that the concept of cheating is an unproductive frame for academic integrity, and we should instead re-centre the concept of “validity” in assessment. Separating the ethical or values-based aspects of cheating – that cheating is wrong or dishonest – from the assurance of learning means we can avoid the “fundamental attribution error” of ascribing cheating to a student’s individual, unethical choice. Instead, we can look for ways in which the system itself might be “wrong” and not just the student: are the methods of assessment such that “all capable students can complete [the task]”? (p. 7)
For Generative AI, this paper offers a useful perspective: the use of AI is not inherently good or bad, wrong or right. It “becomes unacceptable when it threatens validity” (p. 8), but that does not preclude its use in all tasks. Nor does it mean that “No AI” (Level 1 on the AIAS) is necessarily a judgement on whether AI is ethical or not: it is simply an acknowledgement that, in some situations, for some assessments, the technology cannot coexist with the educator’s judgement of what the student can or cannot do unassisted.
In our original paper, we argued that for “No AI” assessments to work, they most likely have to be conducted under supervised, technology-free conditions: not necessarily exams, but certainly on site and in person. This is for two reasons. Firstly, there is no way to guarantee a student with access to a device (phone, laptop, tablet, pair of Meta Ray Bans…) does not have AI assistance. Secondly, if we attempt to use detection tools or other technology solutions, we run the risk of creating equity issues, for example between students who have access to more sophisticated, paid AI products (and who will get away with using them) and those who only have limited access to free tools (and will likely get caught).
Dawson, Bearman, Dollinger and Boud have a clear stance on this issue too:
assessments that depend on students not using artificial intelligence but are incapable of preventing students from doing so, are not particularly useful for high-stakes assessment of learning. (p. 8)
No answers, but progress in the right direction
Ultimately, the advice from TEQSA, the paper from Dawson et al., and our own AIAS offer little in the way of “stopping” cheating. They all, however, point to a necessary reframing of academic integrity that could benefit all students.
Cheating behaviours are nothing new, and generative AI may have contributed to the rising number of students finding ways to bypass learning in their university courses and in K-12 education. But we also need to acknowledge that “cheating” is not just a student-issue; it’s also a systemic problem that reflects issues with our assessments.
Hopefully, these technologies won’t be seen simply as a threat to academic integrity and learning, but as a way to shine a light on some of those bigger picture concerns.
Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch:
Leave a Reply