Leon Furze

I’m working with a number of schools, tertiary providers, and organisations right now trying to navigate AI policy and guidelines. Some education providers are using the National Framework for Generative AI in Secondary Schools as a guideline. Others are using UNESCO materials, or drawing on their own research. Most have only just started the process, and are looking for clear advice.

One huge issue is the speed of development of Generative AI, especially chatbot technologies like ChatGPT and Bing, image generation like DALL-E and Adobe Firefly, and even video, audio, and code generation. Trying to keep up with each new app and feature is impossible, and sometimes it’s better to hold off than rush in to poorly designed or inappropriate apps and services. Even as I write this post, OpenAI has released another update which will knock out many AI startups, including in edtech. Sometimes it really is more appropriate to wait.

Recently, I even learned that there’s a term for this: the Wait Calculation. The Wait Calculation suggests that, with technology advances, sometimes its better to wait a while for a more advanced technology to come along than to rush in. The example given is space travel: if you were to set off now with current technology, you’ll be passed in a few decades by someone who set off later with better tech.

Sometimes, the reward of being a first mover is greater than the risk of being outdone in the long term. At other times, it’s better to stay on planet Earth and wait for the technology to mature. The same is true with AI.

The Need/Potential Matrix

The Need/Potential Matrix is a tool based on an Eisenhower matrix – you might have seen it for deciding whether things are important, urgent, or some combination.

In this context, it’s a useful tool for deciding whether an AI technology, app, or system is worth diving into now or waiting a while (or not worth it at all). It’s also focused on the needs of your organisation, whether thats K-12, tertiary, or outside of education. It asks: Do we need this technology? Do we need it right now? What will happen to it in the future?

Here’s what the Matrix looks like:

High Future Potential	Low Need	High Need
	Quadrant 1: Low Immediate Need & High Future Potential	Quadrant 2: High Immediate Need & High Future Potential
	Emerging AI technologies that might not be immediately necessary or complete, but hold promise for significant advancements in the coming years.	AI tools that provide instant solutions to current challenges and also have great potential for future advancements.
Low Future Potential	Quadrant 3: Low Immediate Need & Low Future Potential	Quadrant 4: High Immediate Need & Low Future Potential
	AI technologies that neither address a pressing need nor hold much promise for future development: solutions looking for problems.	AI tools that address current urgent problems but might be replaced or become obsolete as technology advances.

Examples of the Need/Potential Matrix in action

Quadrant 1: Low Immediate Need & High Future Potential

Q1 technologies include anything which is on the mid-far horizon but which offers huge potential. It’s a “watch and wait” quadrant that might include technologies like quantum computing, brain-computer interfaces, autonomous vehicles, and advanced robotics.

The chances of bringing these technologies into classrooms directly is slim, but it’s likely that some students will encounter these fields in the future and you could certainly partner with universities and industry to explore what’s possible.

Quadrant 2: High Immediate Need & High Future Potential

In Q2 you’ll find the technologies which are useful right now and have future potential. It’s better to think of fields rather than specific apps, knowing how quickly apps can be superseded. For example, the technology behind apps like ChatGPT – Large Language Models, image recognition, image generation, etc. – are going to continue to develop and are already at a stage where they are useful for many purposes.

It also pays to remember that AI as a field is much broader than generative AI, and that by far the biggest application of AI already in education is not GenAI but predictive algorithms and data analytics. These technologies need just as much attention, understanding, and interrogation.

I’m of the opinion that multimodal Generative AI is the technology worth following here, as it is highly likely to continue advancing over the next few years and will also drive advances in a lot of other areas, including augmented and virtual reality. Multimodal generative AI, including image generation/recognition, code, internet browsing, video, audio, and of course text generation is also right here and right now, and has many immediate applications.

Quadrant 3: Low Immediate Need & Low Future Potential

These are toys and distractions. Technologies which might be driven by hype or marketing, but which don’t have a lot of substance. For all kinds of reasons, I feel that most “chatbots” and web applications built on top of GPT (such as “teacher lesson planning AI assistants” and the like) fall into this category.

These technologies don’t address a real current need (I understand it’s attractive to generate bulk lesson plans, but what teachers really need is more time and fewer administrative overheads to create their own resources) and they don’t have longevity. Most are quickly superseded and the industry is too unpredictable to invest time and money into things which might disappear in a month or two.

Quadrant 4: High Immediate Need & Low Future Potential

These are apps and services which are worth paying attention to, but not building an entire system around. Some chatbots might fall into this category, though I’d suggest they’d have to be solidly built and backed by some serious safety features (which generally means serious money, given how much of a target these systems are for attacks).

ChatGPT itself might fall into this category. It has great potential for immediate use, but OpenAI is yet to create a truly successful and sustainable business model, and a company like Microsoft might ultimately absorb it or replace it. That doesn’t mean it’s not worth learning how to use ChatGPT right now though, since the skills learned on that one platform are transferable to other GenAI applications and systems.

Questions for decision making

To make the Matrix even more useful, you can use the following questions to help judge the Need and Potential of the technology or application under review. Obviously there is an element of human judgement here and you also need some insights into the technology and its future. You’ll also need a solid understanding of the problems you’re trying to solve.

One way to use these is to score each question out of 5, and then use the total score to determine (at your discretion) whether it falls into the “high” or “low” range for Need and Potential.

• Low: 0-23
• Mid: 24-37
• High: 38-50

Questions Based on Need (Each out of 5):

1. Does this technology directly address a current challenge in our institution?
2. Is there a demand from stakeholders (students, staff, parents) for this kind of solution?
3. How does this technology align with our current curriculum or teaching methodologies?
4. Can the technology be easily integrated with our existing infrastructure?
5. Does this tool offer a solution that is significantly better than our current methods or tools?
6. Will this technology reduce workload or administrative burdens for staff?
7. Is the learning curve for this technology manageable for our staff and students?
8. Can the tool be implemented without significant disruption to current processes?
9. Does the technology provide any immediate cost savings or efficiencies?
10. Is there data or evidence supporting the tool’s effectiveness in similar educational settings?

Questions Based on Potential (Each out of 5):

1. How scalable is this technology as our institution grows or evolves?
2. Is there a clear roadmap for the technology’s future development?
3. What are the chances this technology will become obsolete in the next 5-10 years?
4. Can this tool be adapted or customised to meet future challenges or changes in curriculum?
5. Is there potential for this technology to foster new ways of teaching or learning that we haven’t considered?
6. Does the technology have a supportive community or ecosystem for continuous learning and sharing?
7. Are there any emerging trends in education that this technology is poised to capitalise on?
8. How does the technology position our institution in terms of being a leader in innovation?
9. Are there opportunities for collaboration or partnerships because of this technology?
10. Is there potential for the tool to offer long-term cost savings or become more cost-effective over time?

Let’s take a totally unscientific approach here and see how ChatGPT scores itself against those criteria:

Questions Based on Need:

1. Directly addresses a current challenge: 5
   • ChatGPT can assist with a variety of educational tasks, including tutoring, answering questions, and aiding with research.
2. Demand from stakeholders: 4
   • There is growing interest from educators and students in utilizing AI for learning and teaching.
3. Alignment with curriculum/teaching methods: 4
   • ChatGPT can support existing curricula and teaching methodologies by providing information and facilitating new forms of interaction.
4. Integration with existing infrastructure: 3
   • Integration varies; while ChatGPT can be accessed via the web, full integration with educational platforms may require additional development.
5. Improvement over current methods: 4
   • Offers a novel approach to information retrieval and interaction that can enhance learning experiences.
6. Reduction in workload: 4
   • Can potentially reduce administrative burdens by automating responses and generating content.
7. Manageable learning curve: 5
   • ChatGPT is user-friendly and requires little technical knowledge to interact with.
8. Implementation without disruption: 3
   • Implementation is relatively straightforward, but its incorporation into teaching requires careful planning to avoid disruption.
9. Immediate cost savings: 3
   • May reduce some costs associated with content creation and tutoring, but it’s not a direct cost-saving tool.
10. Evidence of effectiveness: 3
    • While there is anecdotal evidence of its usefulness, formal studies on its effectiveness in education are still emerging.

Questions Based on Potential:

1. Scalability: 5
   • Highly scalable; can serve many users simultaneously without degradation in performance.
2. Clear development roadmap: 4
   • OpenAI continues to update and improve ChatGPT, though detailed public roadmaps are not always available.
3. Obsolescence risk: 2
   • Rapid developments in AI mean that ChatGPT could be surpassed by more advanced systems in the future.
4. Adaptability to future challenges: 4
   • The AI model can be updated and trained on new data to remain relevant.
5. Potential for new teaching/learning methods: 5
   • ChatGPT has the potential to foster new educational methods, including personalized learning and interactive education.
6. Supportive community/ecosystem: 5
   • There is a strong community of developers and users around ChatGPT and generative AI.
7. Emerging trends capitalization: 5
   • ChatGPT is well-positioned to capitalize on trends in personalized education and AI-assisted learning.
8. Innovation leadership positioning: 4
   • Using ChatGPT could position an institution as forward-thinking and innovative.
9. Collaboration/partnership opportunities: 4
   • There are many opportunities for educational institutions to collaborate with AI developers and other organizations using ChatGPT.
10. Long-term cost savings: 3
    • While there’s potential for long-term savings, it depends on how ChatGPT is integrated and utilized over time.

Total scores:

• Need: 38 (High)
• Potential: 41 (High)

Nice to see that ChatGPT is optimistic about it’s own future…

If you’d like to discuss Generative AI guidelines and policies or professional development, get in touch via the form below: