Leon Furze

When I work with K-12 schools, the approach to generative artificial intelligence (AI) guidelines generally begins with an audit of current policy documents that relate to digital technologies, such as user agreements, cyber safety policies, and communications. The goal is to identify areas where GenAI needs to be taken into account, without necessarily writing entirely new policies.

Once a broad strokes approach is established, it usually falls back to the assistant principals and directors, particularly people with a role in the school like:

• Deputy Principal
• Assistant Principal Teaching and Learning
• Director of Innovation
• Director or AP of Student Wellbeing
• Head of Faculty
• Year Level Coordinator

While it’s absolutely necessary to have some high-level policies and guidelines, the appropriate adoption of Generative AI is being driven by middle leaders to translate to classroom practice.

Generative Artificial Intelligence in Different Subject Areas

I’ve written before about the importance of faculty-level strategy. In this short series of posts, I’ll be talking about strategic planning for faculties handling artificial intelligence. Every subject will be affected by artificial intelligence differently; what is successful in English will be entirely different in mathematics, design technology, or music. There are also the ethical and practical considerations of the technology that students might come across, which vary from topic to topic.

Of course, there are some transferable skills which might be helpful for faculties to deliberately take on, distributing the load of teaching students how to use generative AI. For example:

• The English faculty may focus on the use of generative AI writing tools such as large language models and applications like ChatGPT.
• The arts faculty may be better served focusing on the ethical and creative implications of image, video, and audio generation.
• Subjects like science, mathematics, and digital technologies may explore the use of generative AI for coding, data science, and data analysis.
• Foreign language subjects might look at multilingual translation tools.
• Health and Physical Education classes may look at the impact of algorithms in healthcare or how AI might be used for medical assistants, personal training, and coaching.
• Design and technologies teachers might incorporate generative AI into various stages of thinking, from the use of large language models for idea generation to next-generation 3D modelling for rapid prototyping.

The applications of multimodal GenAI are vast, and in many fields, we’ve barely scratched the surface of the potential for AI to support the full range of subjects. Until we develop methods for subject experts to grapple with these technologies in their own disciplines, we will stay at the surface level.

This first post lays out a general strategic framework that any faculty leader can adapt to bring discussions of this technology into their domain. In subsequent posts, I will be inviting faculty leaders from across a range of disciplines to contribute ideas on how AI can be used in their disciplines.

Step 1: Review Frameworks and Policies and Establish a Faculty Vision

The first step in your faculty strategic planning process should be to get the lay of the land. If you’re not familiar with key local, national, or international frameworks on artificial intelligence, you should spend some time reviewing these documents beforehand. This should not occupy time in a faculty meeting.

Personally, I’m allergic to having meetings for the sake of meetings. I hate meetings where documents are shared around the table for people to review them, since there’s a tacit assumption that people won’t do this in their own time. Treat your colleagues like adults. If everybody values the importance of the task, rather than holding a meeting to discuss policy documents, create a shared folder and throw in some of the most important ones:

• UNESCO guidelines for generative AI in education and research
• The Australian framework for generative AI in schools (or national equivalent)
• The UK DfE advice
• The VINE guidelines that we produced last year as a springboard into schools creating their own policies
• Finally, if your school has one, its own updated GenAI guidelines and policies

Share these documents with your faculty, along with a brief survey or even just some questions in an email that ask:

1. How familiar are you with generative AI tools like GPT, CoPilot, Adobe Firefly?
2. What are your concerns as a [subject] teacher?
3. What positives may come from GenAI in our subject area?
4. How do you believe this faculty should respond to generative AI in light of these guidelines and frameworks?

Give faculty staff a week or two to respond. Stress that this isn’t supposed to be a time-consuming task and it’s just the beginning of a conversation. Using these responses, create a draft vision for your faculty. Keep it simple (nobody wants to do a post-it note vision board exercise) and direct, something like:

Students in [subject] will understand the ethical implications of GenAI which relate to this field, such as [ethical concerns], and will know how to appropriately use the technologies to support their studies.

Tip: If you collect these answers via survey or email, grab all of the responses and run them through a quality model like Claude 3 (its subscription is $20 USD a month). You’ll get a synthesis of concerns, ideas, and a possible way forward for your faculty. As the faculty leader, you can set the initial vision for your faculty based on this feedback, which you will share in the first meeting.

Step 2: Attack Your Assessments

This activity will take place in a scheduled faculty meeting. You’re going to attempt a variety of your faculty’s assessment tasks from the point of view of a student deliberately misusing AI. The reason: “confronting the brutal facts“.

Primarily, this is to point out vulnerabilities in assessments, but also to move some of your staff beyond the “this doesn’t affect me” syndrome. When ChatGPT was first released in November 2022, it was largely seen as an English teacher’s problem—just that chatbot thing writing not so great essays about Jane Austen. Since then, the technologies have developed rapidly, incorporating image recognition and generation, the ability to address complex mathematics, and increasing competencies in audio, video, and code. If you don’t think your assessments are vulnerable to generative artificial intelligence, this activity might prove otherwise.

To be successful, you need a couple of people in the faculty that are confident using a range of generative AI tools. By now, in most schools that I work with, there are a handful of people who’ve been using text AI or maybe even code, audio, and video generation. Now’s the time to lean into those people. If you don’t have anyone like that in your faculty, you might need to adopt the roles yourself.

Before the faculty meeting, ask staff to share one or two current assessment tasks, either by dropping them into a shared folder or by sharing the link to the assessment in your school’s learning management system.

With your chosen GenAI-using faculty members, give them the instruction to adopt the role of a student deliberately using GenAI to complete as much of these tasks as possible, as quickly as possible.

Depending on your subject area, you may need to get a bit creative. For example, in the visual arts, you could use a variety of image generation platforms to create things like amateurish photography portfolios. Use language prompts in platforms like Midjourney, such as “shot on an old iPhone” or “taken with a cheap digital camera,” deliberately avoiding some of the telltale signs of AI. For example, if you avoid including people, it’s much harder to spot AI-generated images.

In your next in-person faculty meeting, share these examples with your colleagues. You might have AI-generated essays, images, or even audio/video. For the maths and science teachers, you might want to investigate something like ChatGPT Plus’s code interpreter, which gives it the ability to both write and execute code. GPT-4 also has much better mathematical reasoning abilities than most other models. This means it can either use code like python, with its extensive math capabilities, to do the work, or for simple problems just solve them directly.

Here’s an example using the first question from the 2023 VCE Mathematical Methods paper (a senior secondary certificate level study). I screenshot the first question from the paper, drop it into ChatGPT Plus, and then check the answer against the examiner’s report:

Step 3: Small Experiments: Fire Bullets, not Cannonballs

Once you’ve identified vulnerabilities in your assessments and started to explore the potential of generative AI in your subject area, it’s time to start experimenting. However, it’s important not to go overboard and try to “revolutionise” your entire curriculum overnight. Instead, take a lesson from Jim Collins’ book Great by Choice and fire bullets, not cannonballs.

What does this mean in practice? Rather than trying to implement large-scale changes across all year levels and all assessments, start small. Encourage your faculty members to pick one or two assessments or units of work where they can experiment with integrating generative AI tools. The aim now is to either attempt to AI-proof the tasks (which I’ll explain more later, in the Assessment Scale), or to work out which skills are fundamental to the assessment and which processes can appropriately incorporate AI.

Some examples might include:

• Using a language model like GPT to generate writing prompts or to provide feedback on student writing
• Experimenting with image generation tools to create visual aids or to teach students about the capabilities and limitations of AI
• Using code generation tools to scaffold student learning in programming or to automate repetitive tasks
• Exploring the use of AI-generated scenarios in science or health classes

The key is to keep these experiments small and manageable. I would conduct this as a brainstorming activity in a faculty meeting, perhaps the same one as the “attack your assessments” activity.

Tip: If you treat this as a brainstorming activity, then capture those ideas an use AI to refine or expand upon them. Models like Copilot, Gemini, and Claude have image recognition and can successfully transcribe handwritten notes, post-its, and board notes (providing your handwriting is better than mine). You could also use voice transcription software – with everyone’s consent – to record the meeting and transcribe the “verbal brainstorm”. Once you have this data, you can feed it back into GenAI to expand or organise your ideas.

Decide on a handful of these ideas to trial in the classroom, or in faculty planning and curriculum design. As faculty members conduct these experiments, make sure to create opportunities for them to share their findings with each other. This might be through an online discussion forum or email thread, or just informal conversations in the staff room.

Over time, as you start to see patterns emerge in what works and what doesn’t, you can start to scale up your efforts. By starting small and “firing bullets”, you can minimise the risk of wasting time and resources on large-scale initiatives that may not pay off.

Step 4: Update Your Assessment Practices

Once you’ve identified vulnerabilities in existing assessments, and brainstormed a few ways to incorporate AI, it’s time to think about a faculty approach to assessment. The AI Assessment Scale (AIAS) provides a practical framework for integrating GenAI tools into assessments while maintaining academic integrity and fostering student learning. If I’ve worked with your school directly, chances are you’ve seen the AIAS already, and perhaps even adopted it as a whole school approach. Either way, it’s still important to contextualise it for your subject.

The AIAS was developed with Dr Mike Perkins, Dr Jasper Roe, and Associate Professor Jason MacVaugh. The first peer reviewed paper on the AIAS can be found at JUTLP, and we also have a preprint on the pilot study at British University Vietnam.

The AIAS consists of five levels that guide educators in determining the appropriate use of GenAI in assessments:

1. No AI: Students complete the assessment without the use of any GenAI tools. This level is suitable for testing knowledge retention and comprehension, such as in-class essays or multiple-choice exams.
2. Ideas and Structure: GenAI is used for brainstorming and organising ideas, but the final work must be human-authored. This level is useful for idea development and can be applied to subjects like language classes or collaborative brainstorming sessions.
3. AI Editing: Students use GenAI for refining and editing their work, focusing on language improvements and multimodal content. This level is beneficial for subjects like English, where students can use AI to check clarity and organisation of arguments.
4. AI Completion, Human Evaluation: Students actively use GenAI for specific task components and critically evaluate the AI outputs. This level encourages understanding GenAI’s capabilities and limitations and can be applied to subjects like computer science, where students use AI for code generation and debugging.
5. Full AI: GenAI is used throughout the task at the student’s or teacher’s discretion. This level is suitable for assessments where GenAI is integral to the learning outcomes, such as in film and media studies, where students use AI to script and storyboard short films.

To begin updating your assessment practices, start by getting everyone in your faculty to read the free ebook, which provides a comprehensive overview of the AIAS and its applications across various disciplines. You can access a free ebook on the AIAS with over 50 activities for the 5 levels by signing up for the mailing list here.

Once your faculty members have familiarised themselves with the AIAS, schedule a meeting to discuss how the scale can be applied to your specific subject areas. It might also be useful to point staff towards some of my other articles on AI and assessment, which include discussions of ideas like ungrading, authentic and practical assessments, and other forms of assessment which are less vulnerable to GenAI-related misconduct. Encourage faculty members to share their ideas and concerns, and work together to develop a plan for integrating GenAI into your assessments in a way that aligns with your learning objectives and academic standards.

The intention here should be to develop a clear faculty guideline on assessment approaches which can be shared with students prior to setting assessment tasks. You should be able to clearly articulate to students how AI can (or can’t) be used in any given assessment task. Importantly, you also need to be able to explain why AI can or can’t be used. It is no longer possible to simply say “don’t use AI” and hope for the best. As you might have learned in the second step of this process, it is very likely students could use AI in ways which are undetectable, and surprisingly sophisticated.

Step 5: Communicate your Faculty Stance on Generative AI

A strategy is worthless if no one ever hears about it. By now, you should have the following:

• An understanding of local, national, and international frameworks
• A draft vision for your faculty
• Proof that your assessment tasks are vulnerable to Generative AI
• A brainstorm of possible ways to address this as a faculty
• Some tested ideas on a small scale
• An approach to using the AI Assessment Scale in your domain

It’s time to take all of that information and communicate it with students, colleagues, and leadership. This might mean a simple email outlining the steps you have taken so far, or a presentation back to the other faculty leaders. You may choose to put some of your findings to your line manager, perhaps the AP curriculum or Director of Teaching and Learning.

Share the negatives and the concerns, but also your proactive responses, the exciting or interesting discoveries, and how you plan to update assessment practices in your faculty to meet the vision you drafted back in step one. If necessary, update and firm up that vision in a final faculty meeting. Above all else, get your ideas out into the world.

Step 6: Review and Update

Set a schedule for reviewing your GenAI strategy, perhaps once a semester or at least annually. During these reviews, consider the following:

1. Reassess your vision: Is your faculty vision for GenAI still relevant and achievable? Have new developments in the technology necessitated a shift in your goals?
2. Evaluate your experiments: Look back at the small-scale experiments you conducted. What worked well, and what didn’t? Are there successful strategies that could be scaled up or applied to other areas?
3. Update your assessment practices: As GenAI capabilities expand, your assessment practices may need to adapt. Revisit the AI Assessment Scale and consider whether your current approach is still effective.
4. Stay informed: Keep abreast of the latest developments in GenAI, both in terms of the technology itself and its applications in education. Attend conferences, read research papers, and engage with the GenAI community to stay informed.
5. Collaborate with colleagues: Share your experiences and insights with colleagues in other faculties and institutions. Learn from their successes and challenges, and consider how you might adapt their strategies to your own context.
6. Seek feedback: Engage with students and colleagues to gather feedback on your GenAI strategy. Are students finding the incorporation of GenAI helpful in their learning? Do colleagues have suggestions for improvement?
7. Adjust your communication: As your strategy evolves, ensure that you’re communicating any changes or updates to students, colleagues, and leadership. Maintain transparency about your approach to GenAI and its role in your faculty.

Remember, the goal of this review process is not to completely overhaul your strategy each time, but rather to make incremental improvements based on new insights and developments. By regularly reviewing and updating your approach, you can ensure that your faculty remains at the forefront of GenAI in education, and that your students are well-prepared for the rapidly evolving landscape of the technology.

Tip: Consider setting up a shared document or repository where faculty members can contribute their observations, ideas, and resources related to GenAI. This can serve as a valuable reference during your review process and help to facilitate ongoing collaboration and knowledge-sharing within your faculty. Use a platform like Google NotebookLM or a chatbot like Claude with PDF upload capability to synthesise and summarise these shared documents.

Summary

Here’s a summary of the whole faculty strategic planning process, which you should adapt to fit your usual cadence of meetings and the ways you communicate with your colleagues:

1. Review Frameworks and Policies and Establish a Faculty Vision

• Review key local, national, and international AI frameworks
• Create a shared folder with important policy documents
• Survey faculty to gauge familiarity, concerns, and potential benefits of GenAI
• Draft a simple, direct faculty vision based on survey responses

2. Attack Your Assessments

• Identify faculty members confident with GenAI tools
• Ask staff to share current assessment tasks
• Deliberately attempt to complete tasks using GenAI, highlighting vulnerabilities
• Share examples with colleagues in a faculty meeting

3. Small Experiments: Fire Bullets, then Cannonballs

• Encourage faculty to experiment with GenAI in one or two assessments or units
• Examples: generating writing prompts, providing feedback, creating visual aids
• Keep experiments small and manageable
• Share findings through discussions and collaboration

4. Update Your Assessment Practices

• Introduce the AI Assessment Scale (AIAS) to guide GenAI use in assessments
• Familiarise faculty with the AIAS through the free ebook
• Discuss how to apply the AIAS to specific subject areas
• Develop clear faculty guidelines on assessment approaches

5. Communicate your Faculty Stance on Generative AI

• Share the outcomes of steps 1-4 with students, colleagues, and leadership
• Communicate concerns, proactive responses, and plans to update assessment practices
• Finalise the faculty vision

6. Review and Update

• Set a schedule for regularly reviewing the GenAI strategy
• Reassess vision, evaluate experiments, update assessment practices
• Stay informed about the latest developments in GenAI
• Collaborate with colleagues and seek feedback
• Adjust communication as the strategy evolves
• Make incremental improvements based on new insights and developments

I regularly work with schools, universities, and faculty teams on developing guidelines and approaches for Generative AI. If you’re interested in talking about consulting and PD, get in touch via the form below: