Technology is a bit of a Wild West in schools, with individuals and faculties signing up to platforms left, right, and centre. I’ll often look over the various subscriptions a school has accrued over time and find thousands of dollars of “dead apps” – things which people signed up for and never used, or in some cases which are attributed to staff who are no longer at the school.
So in a way, it makes sense to try to consolidate and bring as much technology as possible under one umbrella. With the exception of specialist applications, most staff can do most things within a Microsoft or Google suite, for instance. But there’s a point at which the centralisation of apps becomes unhelpful and transgresses on the autonomy of teachers, and that’s definitely happening with AI.
Worse, it’s not happening because of convenience or cost-saving: the narrative of “one vendor to rule them all” is being pushed by sometimes manipulative narratives of privacy and safety.
Conversations like these matter, because they impact how people learn about and interact with technologies.
Security, compliance and fear
Microsoft’s own documentation leans heavily on the promise that Copilot inherits all the privacy, security, and compliance controls baked into an existing M365 tenancy: isolated tenant boundaries, GDPR alignment, Purview governance, and so on. Partner blogs repeat the message for schools and school leaders, framing Copilot as the responsible choice for cautious leaders.
That narrative achieves two things. First, it paints rival tools like ChatGPT, Gemini and Claude as shady and unsafe, without ever proving the claim. Second, it frames the responsibility of teachers: use Copilot and you’re a responsible educator; step outside those boundaries and you’re careless, maybe even unethical.
Google tries the same trick. In its Gemini for Workspace for Education rollout, the company promises that staff can “chat with Gemini safely and securely” with “enterprise-grade data protection.” FAQs stress that conversations “aren’t reviewed by humans or used to train models.”
Identical verbiage fills vendor pitches everywhere: MagicSchool markets itself as “the safest and most privacy-focused AI platform,” SchoolAI claims it was “built from the ground up for student and data privacy,” and ChatGPT Edu leads with “enterprise-level security” and a promise never to train on campus conversations.
Throughout all these glossy digital brochures, the language of privacy and security does the heavy lifting, authorising adoption before anyone asks whether the pedagogy, price, or performance actually beats competing models.
That’s not how this works…
One of the most irritating aspects of the “privacy and safety” marketing pitch is that it’s based on false premises and misinformation about how Large Language Model-based technologies like ChatGPT actually work. At best, these are lies of omission. At worst, this is companies like Microsoft and Google deliberately misleading school leadership.
Much of the “privacy” discourse around AI alludes to the idea that language models will absorb your data and then regurgitate them at some future point in other people’s prompts.
To put it simply: uploading documents or typing prompts into an AI application does not automatically expose your data to the world.
There are a few things to dig into here. First is the widely held assumption that all AI models “train on your data” by default. Since 2023/2024, and largely due to European Union interventions related to the GDPR, that is no longer true. Many models like Claude, Gemini, and Copilot, by default do not retain training data. It’s not a Unique Selling Point of any particular platform.
For those that do collect training data, like ChatGPT, there are various ways to opt-out. In ChatGPT for instance you can turn off the cynical “improve this model for everyone” toggle in the Data controls setting:
And even if you did opt-in to improving models and providing training data, there are a couple of things to be aware of.
When you submit documents for training a large-language model, they are not stored like files in a searchable database; instead, the text is broken into tokens and absorbed into billions of statistical weights. That optimisation compresses and disperses each piece of writing so much that the original wording cannot be reconstructed, making verbatim reproduction extremely unlikely unless that same text is repeated thousands of times in the dataset.
In practice, the model generalises from your input rather than memorising it, so someone’s future prompt will almost certainly never surface your document word-for-word.
It should go without saying that school staff are not uploading sensitive, identifiable data into AI systems. There is no reason to upload student names and email addresses to ChatGPT, and doing so is likely a reportable data breach if you don’t have those settings tightened up.
But I ask you: are most staff trying to use AI with sensitive information? Probably (hopefully) not.
Telling staff they must use Copilot because it’s “private and safe” when they want to use AI for curriculum design, planning lessons, and administrative tasks is absurd. Even if a language model did train on your school IP and regurgitate it (which it probably won’t), are we really so worried about the intellectual property of lesson plans and school uniform policies that we need to dictate which technologies our professional colleagues can access?
The shadow curriculum of AI
Teachers using AI quickly learn that Copilot can’t match ChatGPT for synthesising lengthy curriculum documents, Gemini for multimodal prompts that incorporate YouTube videos, or Claude for dense text analysis or coding. The result is classic digital technology territory: shadow IT. Staff and students switch to “forbidden” platforms on personal devices, their own wifi hotspots, or out-of-school log-ins. Competing discourses emerge between the official line versus the practices people actually use in their teaching and learning.
If school leadership tries to clamp down (because of course it is another responsibility of school leadership, not the technology companies, to surveil the staff use of platforms), any further critical experimentation retreats from view, and it becomes harder to have open conversations about things like quality, bias, or pedagogy.
Staff and students who use the technology in ways which are not “authorised” stop sharing their experiences, even though they are possibly developing the most sophisticated and approaches and understandings of the strengths and limitations of AI.
Authorised AI and professional agency
Frame any particular platform as inevitable, and debate stops before it starts. Saying, “we are a ChatGPT Education School” or “we are a Google School” stops conversations about the most (in)appropriate technologies dead. Educators are positioned as objects of policy, not subjects with agency, and we swap out professional judgement for fake compliance.
Luckily, there are plenty of ways for school leaders and teachers to resist these discourses, even if you don’t want to resist the broader technology itself:
- Ask for the evidence behind “Copilot/Gemini/ChatGPT is safer.” Is there a documented risk comparison, or only a press kit? Safer than what? Safe from whom? What are the risks exactly?
- Bring “shadow” use into the open through professional-learning sessions that compare tools on pedagogical merit. You might say, “I know that for institutional purposes we are an “XYZ platform school”, but in my personal experience, “ABC platform” is much better for this activity…
- Students and staff need to evaluate multiple GenAI systems, not just the one covered by a licensing deal. This is as much to develop critical understandings of the limitations as identifying strengths.
None of that means abandoning security or safety requirements. There are plenty of “old fashioned” cyber security risks like leaked chat histories associated with AI, just like any digital technology. No one is saying privacy isn’t a concern: but we need rational conversations around these topics, not sales pitches.
Whether you use AI or not is your choice. If a school Art teacher decides that they wouldn’t personally touch image generation with a ten-foot-pole, so be it. If an English teacher decides to go down the rabbit hole and create an entire speculative unit of work that draws on half a dozen LLMs for support, fine. Let staff free-range across a variety of platforms if they choose, especially while the technology is still so new.
Don’t limit the professionals in your schools from evaluating the best technologies for the job based on questionable promises of privacy from companies who, at the end of the day, just want your subscription fees.
Cover image: Jamillah Knowles & We and AI / People and Ivory Tower AI 2 / Licenced by CC-BY 4.0. “The picture uses the metaphor of an ivory tower to show where decision making is done, with a representation of a neural network coming out of it, above a crowd of people’s heads…”
Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch:
Leave a Reply