Dr Leon Furze

This series of posts explores each level of the AIAS in more detail, providing a discussion of different assessment types and activities which might fit at each level. These posts also feature contributions from the education community, and I’d encourage you to join the conversation here and on social media to keep building out the AIAS as a resource.

Level 1 of the AI Assessment Scale, no AI, might seem self-explanatory, but we’ve learned in the past two years that nothing about generative artificial intelligence and education is straightforward. It is not sufficient to simply say “you can’t use AI in this task because AI is cheating”. In order to set an AI-free assessment task, certain conditions need to be met. One of the most important of those will be articulating to students why they shouldn’t use artificial intelligence.

Updating the AI Assessment Scale

It’s easy to fall back on assumptions about what AI will do to learning. There is a lot of hype and speculation in the media and even in academic publishing about the negative impacts of AI on skills acquisition and students’ abilities to learn and apply critical and creative thinking.

But the truth is, this is a new technology, and we do not know the long-term implications. Just as we can’t call AI “cheating”, we cannot justify blanket statements like “You can’t use AI because if you use AI, you haven’t learned anything yourself.”

As with any technology, people will find ways to learn with and through AI, as well as in spite of, and sometimes in opposition to the technology. There will be some skills and ways of learning, planning which benefit from a technology-free approach. Again, that is nothing new. So although Level 1 might seem straightforward, there are some complexities.

Complicating matters further are questions around assessment security. Taking a hard line “no AI” approach may be the most secure in the traditional sense of the word, but it may also open up other problems.

As Phillip Dawson and his co-authors write in the paper, Validity Matters More Than Cheating, there is a trade-off between assessment security and assessment validity when making decisions about locking down assessments, for example, to pen and paper exams. An in-person, supervised, timed pen and paper exam may be more secure than an online or open book exam, but it is also much less accessible and creates many other issues.

So, if you’re going to set assessments at level one, you’re going to need a strong reason for doing so, and a variety of methods beyond the traditional examination.

“No AI” Assessments

Luckily, this doesn’t need to be a conversation about cheating or assessment security. Since the first version of the AI Assessment Scale, I’ve been saying at Level 1, we’re talking about “a bad use of the technology, or a bad use of your brain.”

There will be times where we just want students to demonstrate that they know something for themselves. This could be knowledge or skills. The student could be demonstrating that they have committed to memory the connections between different pieces of legislation or different ways to conjugate verbs, or they could be demonstrating correct posture in a series of yoga movements. In any of these cases, AI will not help.

Students might also be required to demonstrate fundamental skills without using AI as a crutch. For example, can they build a cohesive argument based on a synthesis of different points and then deliver that argument in a compelling oral? Again, for the ultimate performance of this oral, AI is only going to get in the way.

Conducting an AI-Free Assessment

This is where things start to get messy. Although there are many valid reasons why we might want to conduct an AI free essay (and I’ll discuss more of them below), we also have to actually ensure that students are not accessing AI. And this will vary across educational contexts: face to face, hybrid, and fully online modes will have to handle this differently.

In online modes of study, Level 1 assessments would require significant investment of time and resources in technological solutions such as lockdown browsers, webcam, keyboard and mouse tracking, proctoring software and surveillance technologies, a combination of all of these, plus perhaps facilitating the exams via managed devices. This may be just about sufficient to prevent students from using AI. But is this a path that we really want to go down?

I know that it’s not a response many people want to hear, but I think that if you work in an institution that is completely online, and particularly online and asynchronous, you need to acknowledge that students can and will use artificial intelligence in some way.

For those in hybrid or face-to-face contexts, you have the luxury, if that’s the right word, of being able to run technology-free assessments in order to prevent the use of artificial intelligence.

To fully understand “technology free” though, we need to go broad. Here are a few of the technologies currently on the market which can use artificial intelligence:

• Any laptop, tablet or computer with internet access can obviously access AI. If you attempt to block particular websites, there are 1000s of third-party applications that will escape your ban.
• Any sufficiently powerful laptop or device can also run artificial intelligence offline through a locally installed large language model like Meta Llama 3.1 (comparable in many ways to GPT-3.5) or even smaller, lightweight models like Google Gemma and Microsoft’s Phi models. This means that simply turning off internet access does not guarantee “no AI”.
• Any device which can connect via Bluetooth to a mobile phone has the potential to access AI capabilities. Therefore, a smartwatch, smart glasses, such as Meta’s collaboration with Ray-Ban, and other emerging smart wearables, which will no doubt improve over time and offer ways for users to interact with large language models verbally and through voice features.

All of this means that “no AI,” even in face-to-face contexts, basically means no technology. This is okay, as long as students in your cohort do not rely on technology for accessibility. You cannot exclude a student from an assessment who needs to use text-to-speech or speech-to-text technology. You cannot exclude a student from assessment if they have a physical disability which impedes their ability to write by hand. You will need to consider alternatives.

As you can probably tell, conducting a Level 1 assessment significantly narrows your options in terms of how you can conduct the task. I will offer one piece of advice: pen and paper examinations are categorically not the answer. Aside from the obvious exclusion of potentially useful accessibility tools, these kinds of exams disadvantage a wide range of your students. Here are some examples:

• A large, echoing exam hall, typical of mass pen and paper examination conditions, can be hell for a student with an anxiety disorder. But equally problematic could be the echoing walls, the drone of the labouring air conditioner, even the scratching of pen on paper from other students for those with auditory processing disorders.
• Autistic and ADHD students, whether or not they are diagnosed, may find it not only difficult, but physically and mentally distressing to sit a lengthy examination in strict timed conditions.
• Some physically disabled students, including students with chronic pain, may also find it nearly impossible to sit an assessment under these conditions.

One reaction to these concerns has been to say that pen and paper exams will work for the majority and that these other individuals can be treated on a case-by-case basis. This is not, however, a principle of inclusive education. This is still an exclusionary practice which treats disabled individuals as “other” and as problems to be solved, rather than looking for ways to make the entire assessment more accessible for everyone, something which benefits all students.

Options for Level 1 Assessments

Given the technical and practical limitations of no AI assessments, it might seem like I’m suggesting that Level 1 shouldn’t exist at all, but there are still opportunities for diverse, inclusive and engaging assessments which do not use artificial intelligence.

In the previous post on the AI Assessment Scale and our decisions to move away from the traffic light colour scheme, I suggested that the AIAS should be seen as a permissive tool, opening up opportunities. Rather than looking for ways to limit the technology, we should be seeking ways to create opportunities for students to demonstrate their learning and show off their skills. This can absolutely be done without artificial intelligence.

Here are some examples of technology-free assessments which are still inclusive but highly contextual to different subject disciplines:

• In Design and Technology subjects and disciplines where a final manufactured product is involved, it may be possible to conduct a final assessment where AI simply cannot help. Generative Artificial Intelligence is not going to assist you in cutting a dovetail joint, or welding two pieces of metal together.
• In the Visual Arts, artificial intelligence is not going to assist in the composition of an oil painting or the creation of a mixed physical media artwork.
• In Sports, artificial intelligence will not assist with the demonstration of an exercise or the understanding of correct posture, movement, and so on.

Level 1, therefore, can be suitable for practical-based assessments, which have some physical product or outcome as a demonstration of skills.

What about subject areas where the physical product is less immediately apparent? Take my subject disciplines of English and Literature, for example. We have often treated the essay as the apotheosis of the assessment and the only way to reliably judge the students’ abilities. From a Level 1 perspective, we can certainly assess some of these skills through an oral or discussion or debate.

But if we consider the essay or creative writing to be both a product and a process, and even a performance, then we can start to consider ways that no AI assessments for these tasks might work. Students enrolled in a writing composition class, for example, may be required to produce a written essay by hand or typed which does not use artificial intelligence, because the purpose of an essay is to demonstrate their ability to compose an essay. There’s a key distinction here, where the purpose of this assessment is not to demonstrate knowledge through an essay, but to demonstrate a capability of writing an essay.

This does not exclude students who need to use speech-to-text technologies (another form of AI), but it does mean they cannot use Generative AI as part of the transcription process. Powerful transcription tools like OpenAI’s whisper can now capture the spoken word with very high levels of accuracy.

So again, Level 1 assessments are highly contextual. We do not set a Level 1 assessment because it seems expedient or because it appears to be a good way to guarantee assessment security. We set a Level 1 assessment because it is the right choice for the knowledge and skills being assessed, and because it benefits the learner to assess them without the aid of technology.

We do not set a Level 1 assessment as a means of guaranteeing that students cannot cheat, because this is impossible. We set a Level 1 assessment because the educator needs a more reliable indication that the student has learned a particular set of knowledge or skills without AI. This is predicated on an understanding of assessment validity versus assessment security, something which I’m still learning about myself.

From the Community

In this section of each post, I have collated responses from social media, emails, and discussions with K-12 and Higher Education educators about how they would approach each level of the Scale.

I put the call out in a couple of LinkedIn posts, and it’s worth reading (and responding to) the comments directly:

Here is a summary of different “No AI” assessments mentioned in the comments:

1. Shane Fagg (Year 5 teacher and STEM Specialist) suggests that the majority of primary school assessments would fit into the “No AI” category, as students need to acquire core skills first. He notes that students with certain needs might use AI for producing components of assessment, such as creating images for those with physical or fine motor issues.
2. Alex Gray (Head of Sixth Form) provides an example of a “No AI” assessment in a Working Scientifically task, which involves drawing graphs, understanding variables, and carrying out practical investigations. Drawing graphs manually helps students grasp fundamental principles, carefully examine and analyse data, and develop critical thinking skills.
3. Kathleen Naglee adds a caveat that hand drawing might not be appropriate for students with certain disabilities, and suggests that “No AI” assessments should meet inclusion guidelines for assistive technology, as I have discussed above.
4. Laura Dumin (Professor at University of Central Oklahoma) mentions personal reflection pieces, brainstorming, and early drafting as potential “No AI” assessments, depending on the course and learning outcomes.
5. John Dolman (Teacher of Media Studies) suggests that “No AI” assessments are phase-dependent, with tasks such as creative and descriptive writing, expressive and argumentative writing, and discussions on group tasks focusing on soft skills being suitable for “No AI” in later stages of education.
6. Selena Tenakov (UniPrep Schools Coordinator) describes two assignment tasks in UniPrep Schools designed for scaffolding critical reading skills, such as identifying parts and purposes of various resources, choosing evidence, referencing, and time management. These tasks are kept as “No AI” to develop formative skills at the crucial age of 16/17.
7. Matthew Karabinos believes that the foundational stage of any subject matter should start with “No AI,” especially when introducing new topics and skills. This includes writing skills, organising ideas, and new math concepts or skills.
8. Selen Erkan (TED Bodrum College) plans a “No AI” assessment for Grade 7 English, where students write a personal narrative about their own lives, focusing on emotions and lessons learned. This task requires personal emotion and experience communication, reflecting the students’ unique voice and developing skills like self-expression, creative thinking, and emotional literacy.
9. Joanne McGovern (Engineering HUB Manager) suggests that practical assessment tasks are critical for “No AI” in practical subjects, as they assess soft skills and practical techniques while allowing for a project-based learning approach in certain aspects.

Gem Orgallo, an experienced Interior Design educator, also reached out via direct message to advocate for the importance of “No AI” assessments in foundational skills and developing design thinking abilities in interior design programs. She advocates for AI-free assessments in the first semester of year one, which include freehand sketching, technical drawing, colour theory application, paper sculptures, card models, inspiration mood boards, brainstorming, concept development, and other hand model making skills.

Gem believes that AI should be introduced as an extra tool, similar to CAD, only after students have learned these fundamental skills. Her teaching philosophy takes a humanist and constructivist approach, focusing on developing the creative potential within the individual. Gem stresses that the generation of concepts and innovative ideas must originate from a human-centred approach, not from surrendering creative power to AI.

In future posts I’ll be exploring Levels 2-5 of the updated AIAS in detail. Level 1 assessments are going to be difficult to justify and perhaps impossible to fully enforce as the technology develops, and we also want to look for ways to appropriately incorporate the technology. Please keep sharing ideas and justifications for No AI assessments, through the contact form, the comments, and the social media posts!

Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch: