Dr Leon Furze

Large Language Model based technologies such as ChatGPT are currently being hailed as either the saviour or the end of education, depending on which articles you read. The speed at which edtech companies have adopted “AI Magic”, “AI Superpowers”, and “AI Enhanced Mega-Booster Revolutionizers” (OK I made the last one up) has curved exponentially upwards since the release of OpenAI’s chatbot in November 2022.

Foremost amongst the claims of technology companies is the line that Generative AI chatbots will form the basis of a new wave of AI tutors and classroom assistants. This narrative nestles alongside personalised learning, 1:1 study-buddies, and even, God forbid, homework helpers, promising a future where every child has access to a genius in their pocket.

But if ChatGPT is a genius, it’s more like a shambling, incoherent professor than a savvy teacher – more Doc Brown than John Keating.

There’s a fundamental flaw with the idea that ChatGPT or other LLM-based applications are “learning tools”, let alone that they’re going to revolutionise education. As Vance CTO Richard Davies wrote in this post, even if you assumed an LLM to have a perfect store of information and zero hallucinations (which might be impossible), users can only ever reach a surface level of understanding with LLMs since they will either be unable to ask the right questions, or unable to understand the answers.

I responded to that post with my own, where I suggested that trying to learn with a Large Language Model is like stumbling around in the world’s largest library, in the dark, without a torch, a librarian, or even the Dewey Decimal System to help out.

The Dark Library

I love a good analogy – even when I hate them – so let’s run with this one for a while.

When working with LLMs, expertise does not come as standard. The basic premise of training Generative AI is to take an enormous pile of data and learn the “rules”, whether that’s the relationship between words in a sentence, or pixels in an image. As these rules are learned, the training data is reduced into probabilities in a network: the data no longer remains.

Imagine visiting a library, but when you arrive the familiar Dewey Decimal System has been replaced by an obscure and seemingly random collection of rules. It’s impossible to figure out exactly where the strings of numbers are leading you. Should you by chance manage to follow one of these connections to its conclusion, you’re met with just another string of numbers: there don’t even appear to be any actual books…

You’re just stumbling around in the dark. You get the sense that enormous volumes of knowledge lurk… somewhere… in the library, but you can barely even see your own hands. It’s like trying to find enlightenment in a sensory deprivation tank filled with alphabet soup.

The Librarian

As I’m writing this, I’m reminded of the Librarian in Neal Stephenson’s 1992 novel Snow Crash: an analogy which has been applied before to LLMs. In fact, Stephenson also predicted the idea of AI as a “personal tutor” through the Illustrated Primer in another book, The Diamond Age, which he spoke about in reference to ChatGPT in this interview for The Atlantic.

But unlike Stephenson’s Librarian or his Illustrated Primer, current Large Language Models don’t intuitively know what the user is looking for, or how to help them.

Upon entering the Dark Library, there’s no Librarian to be seen. The visitor must conjure one up through an obscure and sometimes esoteric incantation: the prompt. By now, if you have read anything at all about LLMs, you will have heard of prompt engineering. If you’ve gone beneath the surface, you’ll have heard terms like chain-of-thought, one-shot, few-shot, zero-shot, QA…

It’s difficult to know exactly which approach to take with this Librarian. Do you ask a straightforward question, like an internet search? Chat with it like a colleague? Tell it to role play an expert in a field of your choice? Speak to it in code? Whichever conversational style you adopt, you get the unnerving impression that, confident as it sounds, the Librarian isn’t actually searching through troves of information to find you the right answer. In fact, it seems like the Librarian is sometimes just making things up as it goes along.

You still can’t see any actual books either. Standing in this huge, dark, hollow space, you realise that the Librarian is the Library, and a slightly unhinged one at that. You back away while trying to avoid eye contact.

The Problem with Using LLMs for Learning

Although a Large Language Model has learned the “rules” of the enormous dataset it is trained on, it doesn’t actually know anything. It also doesn’t understand, or have the capacity to reflect upon, the connections it has learned. Since an LLM must be prompted in order to generate anything, the user also has a responsibility to ask the right questions in the right manner to get the results they’re looking for.

This makes it very difficult to use a LLM as a learning tool. Learners don’t know what they don’t know, and can’t prompt an LLM to autonomously produce the “right” next piece of knowledge.

Of course, many developers are trying to turn LLMs into something more like Stephenson’s Illustrated Primer – a magical book which contains all the knowledge in the universe, and knows exactly what its reader needs. But since chatbots don’t actually understand anything, they can’t really provide personalised learning. They are still reliant on asking the right questions.

Does this mean that LLMs are completely useless as a learning tool? No, not at all. But it does mean that we need to be more considerate of their limitations as we bring them into education. Instead of presenting them with shiny products which promise the world, we need to help educators understand the fundamentals of Generative AI, and why it isn’t the perfect solution for some aspects of learning.

A highly skilled educator could use an LLM as a versatile and powerful tool, wielding the sophisticated language generation skills to create resources and leverage their own knowledge. In the analogy of the Dark Library, educators provide all of the missing elements: the light, the system, and the understanding needed to navigate the complex corpora of learned rules. They’re the ones who can turn the jumbled chaos of the Dark Library into a coherent, illuminating learning experience.

LLMs are not replacements for human teachers, carefully curated curricula, or the hard work of learning. They’re supplements, not substitutes.

Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch: