Two years ago I wrote an article arguing that Anthropic’s “Computer Use” – then in beta testing – would be the next ChatGPT moment for education. At the time, computer using agents (CUAs) were not common, and Anthropic’s version was strictly limited to research, running only inside a virtual computer. It had restricted access to a few sandboxed applications like the open source LibreOffice, and a limited browser.
It’s taken a little while, but we are now seeing the first commercially available, fully functional computer using agents. With the recent releases of Anthropic’s Fable 5.1 and OpenAI’s Astra (GPT-6), it appears that computer use has, for the most part, been solved.
In this article I’ll run through a few simple examples, most of which were used during the making of this article itself. I don’t think this is Artificial General Intelligence (AGI), but I do think we should be paying attention.
What is a “Computer Using Agent”?
I first came across the term “computer use” in a blog post from Anthropic in October 2024 announcing the release of the new capability. It was made available first to developers and researchers through the API, and described as follows:
Developers can integrate this API to enable Claude to translate instructions (e.g., “use data from my computer and online to fill out this form”) into computer commands (e.g. check a spreadsheet; move the cursor to open a web browser; navigate to the relevant web pages; fill out a form with the data from those pages; and so on).
It caught my attention immediately. I’d already written about the increasing capabilities of AI as a coding tool and the implications of multimodal input and output: for instance the ability of LLMs to “read” images as well as create them.
Computer use combined these functions together: coding capabilities to read, write, and execute instructions in code, and multimodal image recognition to read screenshots of a virtual computer screen and “see” the computer in the way a human user might. But it wasn’t pretty…
Then, in January 2025, OpenAI released Operator, again a research and developer preview of the technology that would later become “ChatGPT Agent”. This release also coined the term Computer Using Agent (CUA), and marked the point at which the major GenAI developers all tilted towards versions of this technology.
I wrote about Operator in February 2025 when access started rolling out to consumer accounts:
In that article, I wrote that:
Within 12 months, you will have access to Operator style AI assistants in every major browser, and on every operating system. I suspect there will be the usual arms race between Microsoft and Google, and that Apple will continue its approach of partnering with existing developers like OpenAI and now Google.
https://leonfurze.com/2025/02/28/hands-on-with-openais-operator/
Let’s see how I went with that prediction.
12 months later…
A year (and a bit) later, and I’m regularly using the Claude in Google Chrome browser extension to do online work, including managing parts of this website, updating course materials on https://practicalaistrategies.com/, and doing various business admin. Microsoft did indeed release a similar feature – Copilot in Edge – though I’m yet to meet a single person who even knows it exists, let alone uses it (much like 90% of all Microsoft Copilot products in the last four years…).
OpenAI released Atlas, an entire browser dedicated to its AI Agent. They later retired the product, simply rolling the features back into the ChatGPT desktop app. Perplexity has partnered with NVIDIA, allowing their Portable Computer to borrow power from multiple devices running local AI.
And in a flurry of activity in August-September 2026, Anthropic released Fable 5.1 and OpenAI released GPT-6 Astra, both of which excel at coding and computer use.
We’ve come a long way since October 2024. These models no longer run exclusively in a safe, sandboxed environment: they’re designed to be deployed right there on your machine. This has caused numerous problems, not least of all the spate of so-called “rogue AI” incidents of recent months.
Computer use in 2026
The combination of improved coding and faster, more reliable multimodal capabilities has led to CUAs that now work for increasingly complex tasks.
As my main AI, I run Claude Code on a headless Mac Mini (in a style similar to the OpenClaw chaos of 2025, though luckily I already had the Mac Mini before they globally ran out of stock…). That device has all my files and folders, and I mostly control Clade Code through a terminal app on my phone or using the Remote Control feature in the Claude iPhone app.
Before we even think about Computer Using Agents, there’s a whole lot that Claude Code can do just running in the command line. For example, on the weekend I set up a small application that allows Claude to create contacts and draft invoices in my accounting platform, Xero. This is possible because Xero, like many applications, has both an API and an MCP connector. We don’t need to get too technical about those things here, but they’re actually more reliable and safer than unleashing a CUA into a browser or computer environment, because they can be given very tightly scoped permissions.
Often, however, there’s a job that can’t be done from the command line. Most tasks are designed for human users and involve navigating graphical user interfaces (GUIs), browsing websites, and clicking buttons. These are all of the things that the tool/multimodality combo of CUAs were designed for.
Here’s an example of two computer using agents: Claude Code running Fable 5.1, and ChatGPT with GPT-6 Astra. In the video below, you can see me using Claude Code via the remote control feature in the iPhone app to drive the Google Chrome plugin. It autonomously opens the Booktopia webpage, finds the correct book, and orders it using the credit card on file.
But as I explain in the video, that’s not the only computer use at play. The video itself was edited by ChatGPT using the desktop editing software Camtasia.
Although the purchase of the book is impressive (and slightly unnerving), I find the video editing much more interesting. This was a reasonably complex, time-consuming task that required a combination of command line software and desktop software. The initial prompt was straightforward:
I’ve got Camtasia open. I want you to make a nice looking video for me out of two video files on my desktop – mac-mini and iPhone, both recorded just now. New project. I want the mac mini video on a black 16:9 background with the video filling most of the frame, and the iphone overlaid on the left hand side and with a rounded border and drop shadowed so it looks distinct. I want them synced up to the same actions. I also want any sensitive information blurred out: credit card details, addresses, the last four digits or expiry date of my card, and so on. By the end, it should look like a seamless, real time video of an interaction between Claude Code running on iPhone and the corresponding actions on the Mac Mini in the chrome browser. Fade from/to black transitions at the beginning and end, and a text overlay of “leonfurze.com” with white text, drop shadow, and transparent background in the lower right – Futura font and big enough to be visible but not too obtuse. When you’re done, export the local movie file to the desktop.
It took ~13 minutes for the first attempt, which is probably less time than it would have taken me. The blurry redaction boxes were a little large, so I asked it to make a second pass. GPT-6 used command line tools to do much of the editing and redactions, and then took the edited screencasts into Camtasia for the layout, transitions, and the overlaid text.
Can AI make me a coffee?
Of course, despite the appeal of using ChatGPT to edit videos, what we’re all really wondering is whether AI can do our laundry, or make the bed, or sweep the floors… Language models are, for the most part, stuck in the digital world, although there are various things underway to give LLM-based AI control of hardware in the real world.
But for now, can ChatGPT at least make me a coffee? In a manner of speaking…
If you’re in the same social media circles as me, you’ll have no doubt seen dozens of videos of people getting GPT-6 Astra to make 3D models, environments, or even entire cities in software like blender and Unreal Engine. I figured I’d give it a go, but my ambitions (and token budget) were a little more modest: I just wanted to see if ChatGPT could render a decent looking cup of espresso.
I opened the ChatGPT app on my phone, used the ‘Remote’ feature to access my Mac, and gave it a simple prompt along with a photo of my morning coffee:
This one took a little while – 23 minutes for version one, and a few more for the final render. Now, there’s no way I could make a complete 3D model of anything at all in 23 minutes, so I’m being a little picky here, but it also burned through tokens, using about 30% of my weekly allowance on the ChatGPT Pro account (call that around $8.33, or a third of the $25 USD/week limit).
The resulting coffee looks almost good enough to drink, or would be if I were also a 3D avatar of some description.
So, why is this important?
I’ll end with a comment that I made back in October 2024, and something from a more recent series of articles about why LLMs are better for code than they are for writing human-to-human communications.
First of all, here’s what I said in 2024 about Claude Computer Use:
Claude computer use is not [a finished product]. It’s highly buggy, comparatively difficult to set up, it’s not free, and in terms of security and safety, this is a world away from even the privacy concerns we had over OpenAI’s product. Maybe Claude computer use should be seen as the GPT-2 or GPT-3 moment – the precursor to something huge. But I don’t want educators to be caught off guard again...
Less than two years on and the functionality of CUAs has reached a point where, safety and privacy be damned, the technology companies have decided to unleash them on the world.
Now, combine that with this statement from my IYKYK series of posts, which I later used in my upcoming book Practical AI Strategies 2:
Everything that happens on a computer happens in code and through software. GenAI is proficient at writing and manipulating code. Therefore, it follows that many operations a computer can carry out can be carried out by GenAI.
We now have a technology proficient in the languages of both humans and computers, and user interfaces which allow us to deploy them in code environments, and the more user friendly digital world you and I are used to. Open source LLMs can already do much of this work, and products like GPT-6 Astra are already commercially available.
Almost any individual task that happens in a digital environment can now be completed by a sufficiently capable LLM. That doesn’t necessarily mean entire jobs, or creative roles, or anything meaningful in a human sense. But we’re at the very beginning of some significant changes to how we live and work in both the online and physical world, and these technologies will play an increasingly complex role.
Practical AI Strategies 2 is out September 9th from Amba Press.
Order your copy now direct from the publisher or through any online bookseller.
Want to learn more about GenAI professional development and advisory services, or just have questions or comments? Get in touch:
Leave a Reply