Recently, I had a Zoom call with a school leader that was supposed to go for half an hour and ran to double that. Admittedly, most of that time was me rambling on, but this conversation covered a lot of the ground I’ve been circling while preparing this series of posts. In particular, we spoke about why schools and universities are still writing policies for a technology that has already moved on. We also talked about what large language models are actually good for (hint: it isn’t learning), and whether there’s any point in teaching students to “use AI well”.
I’m here for the Toyota Corolla…
On a school visit earlier this year, I ran an “ask me anything” panel for staff and the leadership team. Many of the questions that came up were the ones that have been bouncing off the walls of classrooms and boardrooms since the release of ChatGPT. High on the list: how do we write an academic integrity policy for AI?
In this case, like many others, by “AI” they meant Microsoft Copilot and ChatGPT, because those are the technologies they have access to. For most schools and many teachers, those chatbots define the technology.
I can spend half an hour or an hour doing a bit of show and tell, demonstrating what’s happening at the cutting edge: computer-using agents and coding tools, or even AI that interacts with physical hardware in the real world, like my diesel generator. People in the audience are usually impressed, but honestly a little confused. For many, it clearly goes in one ear and out the other, because it looks like a completely different category of technology.
In the words of the school leader I was speaking to recently, they were expecting me to talk about a 1980s Toyota Corolla, and I’ve rolled up in a rocket ship. It’s a nice rocket ship. It looks great, and it gets a few oohs and aahs, but they still kind of wanted to hear about the Corolla.
That school leader, like most, has been working from generative AI guidance written in 2023. They expect that one day the system will hand schools an approved chatbot of its own, like the departmental chatbots rolling out across Australia: NSWEduChat in New South Wales, ClassmAIte in Western Australia, and EdChat in South Australia. When that happens, the response from people working closer to the cutting edge will be: thanks for the 1986 Corolla, but we’ve been using something else for a while now.
We seem to be stuck in a loop of trying to build better chatbots and writing policy about them, while in the real world, large language model based AI has changed dramatically in the last three years. I’ve been saying for a long time that chatbots aren’t the future of AI in education, but the policy conversation hasn’t caught up.
A poor go-between
One of the main issues with the chatbot form of AI, and something I’ve written about several times, is that it makes an incredibly poor interface for communication between humans. This is the AI slop problem I’m sure we’ve all heard about, or at least experienced: articles generated by AI, endless social media posts written by bots, terrible images in chum bucket advertising and even plastered across restaurant menus.
Right now there’s a growing problem, and increasing pushback, against the use of AI for one human communicating with another. This extends into schools. Students complain loudly when they suspect a teacher has created slides or resources using a platform like ChatGPT, and some universities and schools have been publicly embarrassed by their students’ complaints. Academics and educators have been accused of hypocrisy when they’ve been “outed” for using AI in their own publications while simultaneously trying to limit student use.
Most of these errors come from well-meaning experimentation by educators, a phase we’re going through while we learn the best fit for these technologies. The problem, as I wrote in AI;DR, is that it doesn’t matter how carefully the author used the technology. A teacher can draft feedback themselves and use AI to tidy it up, or spend hours curating resources from past years and then condense them into an AI-generated PowerPoint. If it carries even the barest hint of AI, students are likely to switch off. The only thing that seems to count is how the person on the other end receives it.
Of course, those same students will happily copy and paste their own writing into a chatbot and ask it for feedback, and they don’t feel short-changed, because nobody is in the middle. They’re dealing with the technology directly.
This highlights one of the policy issues I keep returning to. AI is an incredibly individualist technology, which is probably no surprise given its Silicon Valley lineage. It comes from an environment where the individual is placed on a pedestal above all else (except, of course, the mass aggregation of individuals into data for the companies that run the technology, but that’s a topic for another post).
I run a business of (almost) one, and I can use Claude Code and GPT-6 in ways that an organisation never could. Some of that is about risk and procurement, but a lot of it is simply because a larger organisation is full of people. This technology works very well for one person, but gets awkward the minute that person’s output has to land in front of someone else.
All of this might be a historical blip. The technology is new, its fingerprints are obvious, and I expect we’ll all become a little more immune over the next few years as AI increasingly mediates our communications. But right now, it’s almost as binary as it sounds: there are plenty of good administrative uses, and very few good uses where AI produces the thing that one human hands to another.
Interfaces, not chatbots
I first encountered language models towards the end of 2020, and I came at them from a writing and linguistics point of view, because here were machines that could produce words. As an English teacher, one of the first instances I saw of language models out in the wild was students using tools like Writesonic and Jasper, designed for paraphrasing marketing copy, to hand in semi-plagiarised paragraphs. That piqued my interest, and eventually led me towards my PhD.
These days, I’m far less interested in the wordy stuff and much more interested in the codey stuff.
For most people, AI means ChatGPT, and ChatGPT is a chatbot that looks a bit like a search engine. But try a different mental model. A large language model is a very, very sophisticated predictive text system that can translate competently between any languages sufficiently represented in its training data. Those languages include not only human languages like English, but also code. So large language model based AI can translate proficiently between humans and code.
Now count the devices in your day that run software and have an internet connection. A sufficiently capable language model can potentially interface with any of them. As I wrote about previously, I can be anywhere in the world, open Claude on my phone, and kickstart the diesel generator on the farm, because the generator is connected to the battery cabinet. The cabinet is connected by a data cable to the house. The house is connected to the Starlink modem, the modem to my home office, and my home office to Claude.
When we write AI policies about chatbots from 2023, we miss this bigger picture. We focus on limiting or constraining the use of AI for human-to-human communication, and we neglect the much more important aspect: human to computer.
Of course, most people couldn’t care less about the code and software side of AI, and fair enough. The world would be a very boring place if we were all preoccupied with whether AI can write proficient code. But as I wrote in the previous article in this series, every tech company working with AI is trying to design the killer app that addresses exactly that problem. OpenAI’s dots and Meta’s Muse, along with a host of other “agentic” AI assistants, are early attempts to take the high-end capabilities and bury them in consumer products. At some point, one of these companies will get it right.
When that happens, AI will likely go the way of the internet and telecommunications networks: a system so big it’s almost impossible to critique, let alone constrain in a single policy. If we’re still writing school policies about chatbots by then, we’ll be in trouble.
You don’t teach the internet
In a session recently, a secondary chemistry teacher asked how we teach students to use AI responsibly. My reply wasn’t exactly what they were hoping for: AI is an inherently irresponsible technology, and a chemistry teacher should probably spend most of their time and energy teaching chemistry.
Maybe my answer was a bit flippant, but AI is an infrastructure-level technology, and the trouble with teaching infrastructure is that it’s so broad you end up trying to teach everything. Imagine a subject called The Internet. I don’t mean digital literacy as we usually do it, which is mostly apps and platforms: open a browser, check your sources, this is how you send a polite email… I mean the internet. How does a browser work? What’s a protocol? Where does data live? What’s an undersea cable for? What is HTTPS?
There’s hardly any room in the curriculum to teach those concepts, and nobody seriously proposes making room. I want you to start thinking about AI in the same way. Schools need a deep awareness amongst staff, first of all, of what the technology really is, and of how it affects the disciplines and industries they already teach towards. In the same way a chemistry teacher might use the internet in their teaching, and direct students to certain apps and sites that are better for some content than others, AI can be part of the background machinery of chemistry instruction. But that teacher doesn’t need an extensive knowledge of language model architecture, or passing trends like so-called prompt engineering.
Of course, I’d still teach AI ethics explicitly. I’d teach the critique of the industry around AI, and broad, solid digital skills. But I wouldn’t spend curriculum time teaching students how to use AI better.
Frankly, the students who want to get their hands dirty with the most sophisticated forms of AI probably already are, and the technology itself is very good at teaching you how to use it, if you’re interested. When I work with AI, I often ask the model at the end of a project to reflect on the decisions and hurdles along the way, and to store a memory of potential improvements for next time. It’s a habit worth developing (processes are more important than prompts), but I don’t see where teaching that habit fits in a Year 9 Health and PE class.
We have to accept that most students are about as interested in the technical layers of AI as they are in the hypertext transfer protocol. If the technology does what they want it to, that’s good enough for most people.
And this is the tension you can probably tell I’m struggling with. Not only do I want to show people the rocket ship instead of the Toyota Corolla, I want to peel off the plating and explain the wiring. But it’s overkill.
A few brutal truths
None of this conjecture about the future of AI, or about sophisticated agents versus chatbots, makes the academic integrity problem go away. We’ve reached a point where, as far as traditional knowledge work is concerned, anyone can use AI for anything. The models on the market now can complete the knowledge aspects of the entire K–12 curriculum, and students have access to them 24/7.
If you want to know whether a student can do something, I think you need to watch them do it, or at least have them talk to you about it. If you want them to do something without AI, you need to observe them doing it, completely technology-free. That will become an increasingly difficult and problematic idea as AI-enabled technologies further enmesh themselves in our daily lives. Otherwise, you have to assume that students can, and possibly will, use AI.
A tempting response, and one I suggested myself in the earlier days of the technology, is to write rules about how to use it well in a given task. Here’s how you might use it for brainstorming in science, or research in the humanities. Here’s a good way to use AI for drafting a paragraph. And here’s something to avoid in a music theory lesson. But when you do that across a whole school, everyone ends up teaching AI instead of their discipline.
And when you break assignments down to the task level and start instructing students about this and that kind of AI use, authentication becomes even more of a burden than it already is. Broadly speaking, the authentication processes of most assessment authorities across Australia require teachers to guarantee that submissions are the student’s “own work”. I think AI forces us to problematise what “your own work” actually means.
So what do we write policy about?
We’re in an awkward in-between period. The most powerful uses of this technology are messy, risky, and carry a reasonably high technical overhead. The easy consumer-facing versions are chatbots, and chatbots are primarily being used for problematic human-to-human communication. That might be someone using AI to produce a slop article or, in an education context, a student using it to submit a slop essay. Either way, a lot of the tension we’re feeling at the moment comes from AI acting as an intermediary between one person and another.
The individualist uses, where an incredibly capable AI assistant interfaces with digital systems on behalf of a user, are out of reach for most people, and therefore very difficult to conceptualise and write policy about. That difficulty is no excuse for policies that only try to restrain the kinds of chatbots that are likely to go out of fashion soon.
I’ve written in the past that you don’t need an AI policy. You need to consider where AI intersects with all of your existing policies. Despite some insistence from various quarters that not having an AI policy is somehow irresponsible, I think it’s backwards-looking to try to constrain this sprawling, infrastructure-level technology in a single document.
So here’s what I want you to do instead.
Read up on what this technology can really do. Think of it more as a Swiss Army knife for computers than a human-to-human communication tool; more rocket than Corolla. Think about the implications of a technology that lets you speak in your own first language to a machine, any machine. Consider the privacy implications, the risks, and the potential of a technology that can automate any software and, potentially, any future hardware.
Stop trying to erect fences around the text-in, text-out chatbots that students are currently using to mass-produce homework, and start thinking about what the system around homework needs to look like if we accept that this technology is our new infrastructure, our new internet.
Anchor your position statements to things that will outlast the near future of the technology: your values, your ethos, and the kind of people you want your students and your community to be.
In the next posts in this Beyond Chatbots series, I’ll explore more practical suggestions, with examples of what schools around Australia and internationally are doing to push beyond the 2023 mindset. Stay tuned!
Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch:
Leave a Reply