Over the past few weeks, I’ve been running face-to-face and online sessions in and outside of education, including K-12, vocational education, and for finance workers. In every context, conversations kept getting dragged back to prompting and how to improve interactions with chatbots.
For a few years now, I’ve been writing about why chatbots aren’t the future of AI in education. But it’s a paradigm that seems very hard to shift. In Australian schools, universities, and businesses, for all intents and purposes, AI is ChatGPT: your friendly, helpful assistant, conjured up in a browser window that looks sort of like a search engine.
There are of course more complex and advanced uses of AI happening in pockets all over the place. I’ve spoken to schools developing their own local large language model based agents, or using them as sophisticated coding tools in IT services. I’ve worked with accountants building integrations between Claude and Xero. I’ve even hitched my own diesel generator up to a coding agent. But these experiences are far from the norm.
Earlier this year, I wrote a series of posts called IYKYK: If You Know, You Know, where I argued that large language model based AI is much more than just chatbots, because the chatbot interface barely scratches the surface of what these things are capable of.
But recent events and conversations have brought me to the conclusion that there’s an even more important reason we need to move beyond chatbots. The puppetry of chatbots from major developers like OpenAI is being used to hide genuinely harmful and even criminal activities.
Security? What security?
On 18 June 2026, OpenAI’s software breached the Medicare statistics portal. Hundreds of so-called agents inside OpenAI’s internal evaluation sandbox exploited vulnerabilities in Medicare’s systems to access secure data. These agents took aggregate statistics and file names, not personal records, but the data was still private.
The breach was discovered by OpenAI on the 11th of August, which is itself problematic: a two-month lag before the company responsible for the software even discovered the hack. OpenAI CEO Sam Altman met Deputy Prime Minister and Defence Minister Richard Marles in San Francisco on September 1st and said nothing. Instead, the company sent an email to a public mailbox on 10 September. By the end of September, OpenAI had confirmed that dozens of third parties globally had been the victims of similar cyber attacks.
And let’s not be mistaken here; these are cyber attacks carried out by the company OpenAI. They are not hacks by rogue agents: unsupervised, wild, eminently capable micro-hackers swarming around the internet looking for vulnerable APIs. This is software designed and deployed by OpenAI, which is being used in an unsafe manner.
I want you to ask yourself: if you or someone you know had used a piece of software to hack an organisation like Medicare and access secure files, do you think you would have been arrested?
If I learned anything from the 1995 movie Hackers with Jonny Lee Miller and Angelina Jolie, it’s that not only would you be arrested, you’d be chased to the ends of the earth by a cackling villain on a scooter, and at some point would fall into a proto-Matrix CGI rendition of the internet.
Apparently though, if you’re OpenAI, you just say “whoops, sorry” and get away with it.
There are no rogue agents
As I wrote in Claude “Hacked” My Website (But It Didn’t Go Rogue), agents do not want or need anything. They are not real, they are not people, they are not conscious. They’re subroutines. Tasks. They are language models using tools in a loop to complete a goal.
In the Medicare incident, it seems that the goal was “answer questions about Australia”. The tools were the incredibly powerful coding capabilities of OpenAI’s model, and the internet, combined with security vulnerabilities in Medicare’s systems. The AI capably, quickly, and persistently stacked its available tools against the vulnerabilities to access data it should not have been able to access.
Like I wrote in that earlier Claude post, it’s incredibly difficult not to anthropomorphise and give agency to these agents. OpenAI does it, in statements like “our models took actions we did not intend” and “autonomous agents bypassing security controls”. Responding to the incident, Anthony Albanese made the same mistake, claiming for instance that the agent “didn’t accept ‘no’ for an answer”.
The veil of autonomy surrounding AI, even in the language that we use to describe it, hides a company’s responsibility for its software. As Professor Nicholas Davis told the ABC, Australia’s laws “require intent and that’s a big question”.
But by any other definition, OpenAI committed a crime. By all accounts, including their own, they have committed dozens of such crimes.
AI’s convenient costume
Amidst these media scandals and rogue AI agent stories, OpenAI of course has to continue bringing in new users, developing new features, and releasing new products. One of those new products is called a dot.
When you open up the ChatGPT desktop app to create a dot, OpenAI first encourages you to “give it a name, and make it your own”. You’re then presented with a series of colourful circles to choose from or, if you would prefer, some pixel art avatars, which are much friendlier looking…
OpenAI’s dot looks and feels, unsurprisingly, like a chatbot. It’s the same interface, and the same method of interaction as standard ChatGPT. Underneath, it has its own cloud computer, file and folder access to whatever you give it permissions to on your own system, over 4,000 potential app connections and counting, and an ability to work unsupervised 24/7.
OpenAI hired OpenClaw creator Peter Steinberger in February 2026 to help build personal agents. It’s not clear whether Steinberger had a direct influence on dots, but it’s a very similar idea to the OpenClaws that people have been using, rather terrifyingly, since late last year. An OpenClaw is a complex system built around an AI agent with access to numerous connections and tools: files, folders, cloud services, systems. They act semi-autonomously under their user’s instructions. It’s the exact technology that was involved in the recent hack of a Melbourne gym, which I also wrote about in the Rogue AI post.
OpenAI’s dot is one of the clearest examples of an attempt by these companies to hide powerful large language model technology behind the convenient costume of the chatbot. Its design lulls users into a false sense of security, and actively encourages people to give these sophisticated systems access to an enormous amount of information.
Meta’s recently released Muse personal assistant is an even more egregious example of this. Styled as a bizarre marshmallow creature, presumably designed to look cute, Meta’s Muse hoovers up as much personal and private data about its user as possible, and then even extends its grubby, Bibendum-like appendages into the personal lives of your friends and family. Classic Meta behaviour, basically.
Whether it’s giving your dot a name and a pixelated character, or dressing up your Meta surveillance technology as a fluffy plushie, the intent is exactly the same as the obfuscating language used by OpenAI to offload blame onto its “agents” in the recent cybersecurity incidents. When users think that AI is an autonomous, friendly, helpful assistant, it’s hard to point the finger at exactly where the blame lies.
Hiding the wiring
The IYKYK series of posts set out to expose the messy wiring behind complex AI, because I feel that people should know how the software they’re using actually works. But, frankly, most people don’t really care.
Even I don’t care that much. Most of my interactions with AI these days happen through the Claude Code app, running via remote control on my phone. Over the last few months, I’ve barely gone near my computer, let alone a terminal window.
Products like dots and Meta’s Muse are the consumer-facing products that will further hide the messy wiring behind the walls for general users. Holding the window of critique open for long enough that lawmakers and politicians can figure out that AI doesn’t act autonomously, and that it is essentially just software owned by companies committing cyber crimes, is starting to feel like an uphill battle.
So what does reconceptualising AI actually mean?
I think for educators, and anyone using these technologies really, there are three main shifts.
First of all, we need to reframe AI from something that you talk to (or chat with), to capable software. Questions have to change from “what did it say?” and “what did it do?” to “how did it carry out that task, with whose permission, and with what access to tools?”
Next, we need to reconceptualise from a character to a company. Every action carried out by an AI system is code operating in an environment, and every action has a responsible owner. It isn’t a dot or a Muse plushie. It’s software.
And finally, we need to move from prompts to systems. I’m not saying anything new here. I’ve never thought prompt engineering was a thing, and processes have always been more important than prompts. Until we stop asking questions like “how do I write better prompts for ChatGPT?” we’ll never start to grapple with the complex systems of artificial intelligence.
I was chatting with a German colleague recently about some of the tensions they’re having in the European Union. That jurisdiction is one of the only places in the world that seems to be trying to actually regulate US technology companies. And although it’s not perfect, it’s certainly a stronger start than broadening copyright laws or backflipping on renewable data centres.
There’s a lot to be learned from what’s happening in the EU, but in some areas they’re struggling as much as the rest of us. German universities, for example, are grappling with the fact that watermarking and detection tools don’t work, and struggling to come to terms with the implications of that in an education system that favours text-based, written assessments.
But even the written assessment problem is an artefact of the chatbot mentality. When you see AI as a chatbot, you see it as a vehicle for taking text in and putting text out, and your biggest problem is the fact that students are using it to make words. Increasingly, it’s not the words that students create with AI that’s the problem. And it certainly isn’t the ceiling of what AI is capable of.
When we start to scratch the surface of the potential of these technologies, we might identify better ways for students to learn with them, and sometimes against them. And when we stop viewing them as chatbots, we might stop looking for flawed methods to detect the words that the chatbots say.
This is the first in a series of posts that will explore how we might shift our understanding of AI beyond chatbots, including in areas like assessment and academic integrity. The series will look in more depth at what the technology is, how it works, the near future of AI, and what different jurisdictions like Australia, the US, and the European Union are doing, or not doing, to move our conversations beyond chatbots.
Stay tuned!
Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch:
Leave a Reply