Leon Furze

Feature image by DALL E & Leon Furze

I had the pleasure of speaking with Mike Sharples, coauthor of Story Machines, which is available now from Routledge. You can also check out Mike’s AI story writer at https://story-machines.net/ if you want to try your hand at writing (or co writing) an AI story yourself.

Transcript (courtesy of Otter.ai)

Leon Furze 0:05
Alright, so I’ve got Mike Sharples with me here this afternoon. And it’s it’s 8pm here in Australia. And that must mean it’s around 11am in the UK mark.

Mike Sharples 0:16
Yep, just 10 past 11 In the morning in the UK.

Leon Furze 0:20
And Mike is one of the authors of this book story machines, how computers have become creative writers. And I recently did a review of story machines on my blog. So I have a vested interest in this. I’ve applied for a PhD in the area of artificial intelligence and writing. And when I saw this book came out, I think I heard you mentioned on a podcast, I jumped straight online and preorder the copy of it. And I’ve really enjoyed the book. And Mike has kindly agreed to come and talk to me this afternoon, and share a little bit of insight into how the book got created, and his own journey into the artificial intelligence rabbit hole, and where he’s heading next. So Mike, could you first of all, just give us a little bit of general history on your work in artificial intelligence and writing?

Mike Sharples 1:11
Okay, well, it goes back to my PhD. So I mean, it’s great to hear that you’re starting a PhD. This is where it all started. For me, that was in the late 1970s, now at the Department of artificial intelligence, and Edinburgh University. And my PhD was on cognition computers and creative writing. So it was designing tools with computers were available at that time to help children develop their writing abilities. And then in the late 1980s, I ended up at COGS, cognitive computing sciences at Sussex University, which was a fascinating, amazing department, because it brought together a whole range of different people exploring creativity and cognition in computers and in humans. And my interest was in developing sophisticated tools for writers based on what we know about how people write. And I became fascinated by the idea of AI systems that can generate narratives. And so I set up a set of studio projects, and then recruited a talented PhD student called Rafael Perez y Perez, who developed a system called MEXICA, which was based on a model of creativity in humans; so, how humans write. And he used that model as the foundation for his MEXICA system, which generates stories about the ancient inhabitants of Mexico. And he’s still working that system 30 years later. So that was how it all started. And around 20 years ago, now, we decided we’d write a book together, and never got around to it until the pandemic came. And we decided it would be a lockdown project. So we work together on the story machines book, which has just been published by Routledge.

Leon Furze 3:17
And I think that lock downs forced a lot of people to take on some of those projects that were on the back burner for a while. You mentioned your own history goes back to the 1970s, which is obviously in those very early days of computing and artificial intelligence. But the work in the book actually goes back a lot further than that, doesn’t it? And one thing that I found really interesting because I was reading Story Machines was just how much history there is behind people trying to create artificial intelligence writing machines. And we might not call them that, and they certainly didn’t call them that the time. But are you able to just share a little bit about that research and going back through time, and the kinds of artificial intelligence writing generators you came across?

Mike Sharples 4:10
Writers have been fascinated for centuries by the idea that a machine might act as a writer and whether humans as humans, we are writing machines, whether we are slaves to language, whether we are just machines to produce language. But the first practical machines for generating language came in Victorian times in the 19th century, and particularly, a machine clockwork powered machine, developed by John Clark, in 18, exhibited in 1845, that generated Latin verse and could output over 28 million different Latin verses. And at the time, there was a lot of speculation about thinking machines. He was a contemporary of Charles Babbage, who developed the difference engine and the analytic engine. But that was an attempt at creating a machine that would write poetry. With the first digital computers in 1951, the very earliest ones was Manchester by Alan Turing and a colleague of Turing’s John, who wrote a program to generate flowery love letters. And you can speculate on why that was the case. And then from the 1960s onwards, there were a whole series of projects to produce machines, programs that would generate stories. And then in the first book, amazingly, that was published, written by an AI assistant was in 1993, called Just This Once and by a guy called Scott Trench, who worked with an AI system that he had been developing for eight years to publish a novel in the style of Jacqueline Susan Lee a potboiler novelist at that time. So there’s quite a history going back over 100 years of designing systems that will generate language, poetry, and more recently, stories and even books. The big change in the last year really has been neural network AI systems called transformer AI systems that can generate plausible text on any topic. So unlike, say, Scott French, who took eight years to design his expert system for generating in the style of Jacqueline Susan, the new systems will imitate any style. And they will also write order, you can give them say an essay topic. And it will, at the press of a button, generate a 2000 3000 word essay, it will generate an academic article, they will translate texts that will summarise scientific works. So the general purpose language machines that are hugely fluent, but as we’ll see, have some fatal flaws.

Leon Furze 7:19
Now I’ve, I’ve played around with a number of very sort of entry level natural language generators. And the output is a bit clunky at times, but possibly human almost. I teach students in year seven to 12, so sort of 13 years old and upwards. And I also do a bit of assessment and lecturing at the university level. And I’ve certainly seen work coming out of these AI writers, which is comparable to some of those, some of those real students, and I’ve certainly seen some of the real students using work, which is almost definitely been generated by those AI writers. So our students are very well aware of these these programs. What do you see those that as the biggest advantage of this technology, we can talk about the pitfalls and some of the risks a little bit later. But for now, what do we stand to gain from these these big leaps forward for writers?

Mike Sharples 8:20
I think there are two main advantages. One of them is that it provides a new tool for writing for both beginning writers and also professional writers that focuses on creativity. So what they offer is the opportunity for the human writer to explore accuracy, argumentation, creativity, and let the AI handle the expression, the production of the words. And that’s different to previous tools like spellcheckers of resources, where the focus was always on the surface text. So they allow a new way of exploring different forms of expression. For example, different continuations of a story, or different argumentation forms, while allowing the system to take care of the work production, so I think they could be valuable tools for actors. The other is that they some sorts of AI, particularly the earliest symbolic AI, language generators, can act as models of the writing process that can be tested, inspected, modified, so that if you can build a system that acts as a story generator, you can then get inside it and see how it works in the way that you can’t get inside a human brain. And you can try modifying it, you can try ablation experiments. So what happens if you take something out of it, how will that change the way it writes? And so it’s a way of exploring the creative process. So, you know, weather forecasters and economists have models of weather patterns or the economy that they can play around with. And they can use the forecasting, we’ve never really had something like that for writing. But now we have a range of different tools that we can use as models of the writing process, to see what makes human creativity, whether it can be modelled in a machine, and then what’s the difference what’s lacking in these models, that really indicates what’s special about human creativity. So I think it’s great to have a model of the human writing process.

Leon Furze 10:50
Yeah, and I found that a fascinating part of the book actually was the, the amount of detail that you go into in the potential for AI to help unravel some of that creative process. So I’m a writer, myself, and I write a bit of short fiction as well. And, you know, there’s complicated ideas that are kicking around in my mind whilst I’m writing, but during the actual process of writing, and this is common, for a lot of people, you’re almost in that kind of meditative state where the words just come out and end up on the page. And then you have to go back and very consciously edit, to be able to unravel some of that meditative aspects of creative writing and find out what’s going on under the hood, I think would be, would be a fascinating line of study. And also probably really helpful for people in terms of developing their own writing skills, if you can more fully explore and articulate what’s actually going on there behind the eyes when somebody is fluently writing, to be able to then use that and then provide that back to them, I think would be really, really fascinating.

Mike Sharples 11:56
Yeah, exactly. Where is the why some professional writers have said that they see themselves as writing machines, because they don’t understand how the words come out of their heads, and being able to protect in, in some sense of sort of a download or a simulation of your own writing process. And then, not only to see how writers work in general, but perhaps to see how your own writing process operates. And to change it and to adapt it into play with it’s it’s a fascinating playground, particularly for people who are either professional writers who are interested in the writing process.

Leon Furze 12:42
I think, I mean, for me, personally, that sounds very exciting. I’m sure there would be detractors who would would say that we shouldn’t be delving into the creative process, and that the creative process should remain a mystery. What do you think of some of the biggest risks potentially, on the horizon for this technology?

Mike Sharples 13:04
So I think that currently we’ve got two sorts of writing systems at the moment the traditional symbolic AI systems, which are based on models of the writing process, and these new transformer systems, and the new transformer systems are expert wordsmiths, that can write really plausible text texts that can be convincing. They can generate an entire blog or a student essay. But the problem is that they have no internal model of how the world works. They have no representation of causality, of the accuracy of. So what they can do is they can write text, but they can’t understand what they’re writing. They can’t introspect on their own writing, which means that you have to take anything that these systems produce with a very critical eye. And the danger is, of course, that people who read them, and students will just take them at face value, they will take what they produce at face value. And many of the texts produced by these systems are just simply accurate. If you ask it to write an academic article, which I’ve done, it will look plausible but it will be riddled with inaccuracies, it’ll just make up facts. It will make up research studies, it will invent references, it will produce conclusions that don’t match reality. And so you’ve got to take everything that they write with a very critical eye. And it’s a good writing exercise with students to get them to generate text with the systems and then to critique them. But the danger is that students will just use them in another thing. Keep away, or that readers will read what they’re produced in an unthinking way and assume that they are accurate and they’re not. Now, there are opportunities to put the symbolic AI systems and models of the mind together with these new language generators and to develop a new generation of systems that combine the two. And there’s work being done on that. But we’ve we’re not there yet, it’s going to be some time before we can have systems that can generate plausible text. That’s also accurate.

Leon Furze 15:34
Yeah, I’ve found in my own experiments, I’ve attempted to run off a few essays about Pride and Prejudice, which is our current senior English text that we’re studying. And, and they started off reasonably plausible, but the the AI writers tend to trip themselves up, then the after a certain period of time, you know, the first few 100 words, they start to go down the road and sort of invented rabbit holes, we had a lot of interesting moments where it’s the essays, were talking about Regency England very broadly. But then we’re creating entirely fictional characters within Pride and Prejudice and having them go off and form their own relationships with one another, and then analysing those relationships as if they were real. So for a student, potentially, quite tricky. If they’re trying to use these AI writers as a co writer or a support, they’ve actually got to be really critically minded when they’re using that software.

Mike Sharples 16:32
And that’s a both a positive and a negative aspects. So it does, you know, it encourages and you can encourage your students to read critically, by generating text with these, and to understand what critical leading is that, you know, it’s not just AI systems that can generate inaccurate text humans can. And so, to encourage critical reading, and to do it with systems that invent stories, and more blatant way, is perhaps, you know, a good exercise with students. But, you know, you need continually to be careful with them. And, yeah, I’ve tried generating essays, generating academic texts. And it is very concerning, because, you know, things like the references that they generate look plausible. And you have to then get beneath the surface text to see that they are inventing. And, of course, they were never designed to be accurate. That’s the thing. They were designed as wordsmiths, they were designed as language continuers, they were designed to style imitators that weren’t designed as models of the world. And we really need to understand that if we’re going to use these systems.

Leon Furze 17:52
And so assuming that we’re not just all going to use them to cheat on our essay writing, what do you think is the biggest implication in education for these writing machines?

Mike Sharples 18:06
Well, the first thing is that they’re, as you say, students are already using them. And it’s a kind of alternative history of education technology, where students at various times in history have had powerful technologies in their own hands. So mobile phones is an example, where, you know, in the early 2000s, students have mobile phones, which they took into school. And the reaction from education systems is almost inevitably first to ignore, and then to resist, and then perhaps, to adapt. And it’s going to be exactly the same with these systems. So it’s going to be very hard to ignore them first, because if you say students are already using them to, to write essays or to help them write essays. The second would be that education systems, schools, universities will try to resist them by setting invigilated exams, saying that students can’t use of these devices in class. But I think the more intelligent way is to then see how you can accommodate them, just as you’ve accommodated other technologies in the past, such as word processor, spellcheckers disorders, and how you can use these creative writing aids. Now there will be times where you want the students to focus on no expression, correct spelling grammar, but there’ll be many other times when you want the students to use these tools as aids to writing to then explore the deeper ways of forming an argument, writing collaboratively. So I hope that over time your education systems, schools, universities, colleges, will come to see them as another valuable tool, and a tool that focuses on creativity rather than just surface expression. But it’s very early days, I mean, it’s only just in the last nine months, that they have come into prominence. And so we’re kind of starting a new academic year with these tools as part of the repertoire of students, and teachers. So let’s see what happens over the next year or so as to how they get incorporated into education.

Leon Furze 20:34
And education is notoriously slow moving beast. I think the best that we can hope for is that the students and the teachers on the ground really will be the ones that are driving the adoption and the use of these technologies. And as I say, you know, we already know that students are dabbling with these technologies in the classrooms. So what I would really like to see is teachers, you know, from, from my perspective, that’s English teachers, but any teachers at all, really jumping on board with these technologies and thinking up interesting ways to use them in the classroom. I’ve got access to a couple of different AI image generators. And, you know, we’re talking the same kind of thing. They’re the the creative options, that the capacity for students to be able to get in there and experiment with prompts and see how these AI image generators change things or can adapt pieces of existing work is fascinating. And the use cases are very, very broad ranging, I think it’s exactly the same for these writing tools.

Mike Sharples 21:34
Yeah, and, and we are very much at the start – it’s one year in, and we’re going to get far more powerful systems, ones that can generate video, for example, ones that can actors, writers, companions. So a back in the 1980s, I was developing writer’s assistant, which, for various reasons, never got very far. But I’m really excited by the possibility now of having a powerful writer’s assistant that can act as a companion for writing, can help with expression can help with creativity. So you know, let’s hope it can go in a positive way. And I think people like you and me, trying to promote the positive benefits of these systems are going to be really important and really influential.

Leon Furze 22:30
Look, I’m definitely optimistic. Now, we were talking a little bit before I turned the recording on and I know that one of the things that’s next on your agenda is restoring the sailboat, but outside of the outside of the sailboat restorations. You’re working on the follow up book with Rafael, could you tell us a little bit about what’s coming next?

Mike Sharples 22:52
Yeah, so we decided, when we started this lockdown project that we’d write two books, mainly because we couldn’t decide which one right so we ended up writing two. So the first one story machines is for a general audience, and particularly for people who are writers and interested in teaching writing. And then the second one is a more technical book. It’s called An Introduction to Narrative Generators. And it’s for computer science students, but anybody who’s interested in what goes on under the hood of narrative generating systems. And so it goes into some detail of different sorts of narrative generators and how they work. And that’s going to come out next year to be published by Oxford University Press.

Leon Furze 23:41
Fantastic. I’ll definitely be looking forward to that one. So thank you very much, again, for your time, Mike. And if you are at all interested in learning more about the history of artificial intelligence, and also just the way that machines can help us to dig in a little bit more around the creative process and what it means to be a creative writer. And whether or not machines can actually be creative writers. I would definitely recommend giving Mike and Raphael’s book a read. And we’ll wait with bated breath for next year for the second instalment to come through. Thank you very much for your time Mike.

Mike Sharples 24:16
And thank you, Leon, thank you for your fulsome recommendation. It’s great to talk with you. Have a good day.

Transcribed by https://otter.ai