Robot revision: Year 12s versus the machines

I’ve been posting a lot recently about AI in education, particularly the potential impact of AI writers in the English classroom. I’m being optimistic too: I don’t think that AI writers are heralding the end of days for human authors. Nor do I think we’re ushering in an age of dubious ethics and constant cheating on behalf of students. But I do think that these programs necessitate a change in the ways we teach and assess writing, so I’ve been coming up with practical applications that could be used in the classroom.

In a previous post, I explored using Mike Sharples’ Story Machine to generate short fables for use in a Year 7 class. The point of that exercise was to prove that AI writers can be used to prompt and augment human creativity, not necessarily replace it. In this post I’m coming back to the essay. I began this series a few weeks ago with an article on the fears surrounding essay writing machines, including their use for “cheating” in assessments. I suggested that when AI writers reach the point of being able to seamlessly present human-level essays – soon – that setting essays simply for the purpose of assessing knowledge will be pointless.

In this post, I’m going to look at a different use case: the AI writer as study buddy, with a bit of friendly competition thrown in.

Problems and opportunities

Because of how these AI writers work, they are excellent at some things and terrible at others. They are great at syntax, grammar, and punctuation: the sentences they produce are often flawless, if a little simplistic. They can also mimic style and voice, which is, again, largely down to the patterns and arrangements of language. But they’re not much good at holding down a story: they can’t keep track of their own errors, they can’t reflect on what they’ve written, and they often go down a bizarre rabbit hole of their own mistakes.

This presents an opportunity for students, who can do all of the above. In fact, the latter skills – those based around consistency, logic, and creativity – are where students have the greatest chance to shine over the machines. SO, here’s the lesson:

  1. Towards the end of the study of a text, when students have sufficient ideas of their own to begin to write essays, get them to write some essay topics. I’ll give examples below.
  2. Feed these essay topics into an AI writer. For the example below I used smodin.io
  3. Working in groups, carefully review the output. I can guarantee some of the following will apply:
    • The essay will be structured “like an essay”, i.e. with an introduction, several body paragraphs, and a conclusion.
    • Unless directly instructed to do so, and using a better quality program, the AI essay will not include direct quotes
    • Spelling, grammar, punctuation and syntax will be largely accurate.
    • There will be great inaccuracies in the actual content. This is because once the AI starts writing, it has no way to “fact check” itself. You might find a confusion of characters, settings borrowed from different texts by the same author, or complete and seemingly random (though they won’t be) errors.
  4. Highlight and annotate both the good and the bad parts. For the good parts, try to find direct textual evidence to support these points. For the bad, try to reach a conclusion on why the machine has made a mistake. Correct the mistakes.

Example: Pride and Prejudice

Earlier this year my Year 12 English class studied Pride and Prejudice, so to test the AI writer I came up with this prompt. Note that the prompt is highly specific to one character and idea, which is a good way to both encourage students to think deeply about the text, and the best way to generate good AI writing.

Photo by Suzy Hazelwood on Pexels.com

Prompt: An analytical academic essay exploring the role of Mr Collins as an expression of regency era patriarchy in Jane Austen’s Pride and Prejudice

Here’s the essay the AI generated, in full, followed by my commentary:

Pride and Prejudice is a novel written by Jane Austen about the various tensions experienced by people in an affluent society. The story revolves around the social hierarchy of that era, with particular attention paid to Mr Collins’s role within it.Mr Collins is introduced as a wealthy man who has recently arrived in England and seeks admission into one of Britain’s most prestigious gentlemen’s clubs, Pemberley. He is Initially rejected on account of his low rank in the local social order (a result of prejudice against landowners), but he eventually succeeds through bribery and manipulation.

Mr Collins is the most prominent character in Pride and Prejudice and his interactions with Elizabeth Bennet are the focal point of the novel. From the moment they meet, Collins is determined to marry Elizabeth and exert his power over her. Bennet responds to Collins’ advances, initially out of politeness, but later out of anger and frustration. Collins’ actions are a reaffirmation of patriarchal norms and values, and as such, have negative consequences for both Bennet and the other women in the novel.

Collins’ relationship with Elizabeth is coercive. From the outset, he puts pressure on her to marry him and makes assumptions about her suitability as a wife. Collins is a master at manipulating the women in his life and uses his position of authority to dictate their behavior. This type of relationship is typical of regency era patriarchy, in which men are in charge and women are implicitly or explicitly subservient.

Collins’ actions have consequences for all of the characters in Pride and Prejudice. Bennet is forced to endure his advances and rejection, and she comes to doubt herself and her own capacities. Mrs Bennet is hurt and humiliated by her son’s behavior, while Jane is despairing and disillusioned. Collins’ actions contribute to the negative stereotypes and caricatures of women that are commonplace in Regency England.

Collins’ role as an expression of regency era patriarchy in Pride and Prejudice has negative consequences for all of the characters in the novel. He uses his position of authority to pressure and manipulate Elizabeth, Mrs Bennet, and Jane, and his behavior contributes to the negative stereotypes and caricatures of women that are commonplace in Regency England.

AI WritingNotes
Pride and Prejudice is a novel written by Jane Austen about the various tensions experienced by people in an affluent society. The story revolves around the social hierarchy of that era, with particular attention paid to Mr Collins’s role within it.Mr Collins is introduced as a wealthy man who has recently arrived in England and seeks admission into one of Britain’s most prestigious gentlemen’s clubs, Pemberley. He is Initially rejected on account of his low rank in the local social order (a result of prejudice against landowners), but he eventually succeeds through bribery and manipulation.Opening sentence is accurate, but not really related to the topic. Immediately the AI confuses Collins for someone else, or possibly several someones. Pemberley is a ‘prestigious gentlemen’s club’, apparently… There certainly were such things in Regency London (and still are), but they’re not a feature of this novel. The entire introduction has developed the theme of social hierarchy but totally missed the patriarchy of the prompt.
Mr Collins is the most prominent character in Pride and Prejudice and his interactions with Elizabeth Bennet are the focal point of the novel. From the moment they meet, Collins is determined to marry Elizabeth and exert his power over her. Bennet responds to Collins’ advances, initially out of politeness, but later out of anger and frustration. Collins’ actions are a reaffirmation of patriarchal norms and values, and as such, have negative consequences for both Bennet and the other women in the novel.Conflating Collins with Darcy? I mean, I love Collins as much as the next Bennet, but he’s hardly “the most prominent character”. The second sentence, however, could be argued, as could the following (albeit vague) statements.
Collins’ relationship with Elizabeth is coercive. From the outset, he puts pressure on her to marry him and makes assumptions about her suitability as a wife. Collins is a master at manipulating the women in his life and uses his position of authority to dictate their behavior. This type of relationship is typical of regency era patriarchy, in which men are in charge and women are implicitly or explicitly subservient.The AI writer seems to have grasped the ‘patriarchy’ aspect of the prompt finally. These points could be developed further by a student, using some evidence from the text to support them. Lady Catherine de Bourgh, in conversation with Collins, recommends that a suitable wife should be “an active, useful sort of person”, and Austen certainly develops a clear and deeply ironic commentary around the “accomplished woman”. This is probably the AI’s strongest and most on-topic paragraph.
Collins’ actions have consequences for all of the characters in Pride and Prejudice. Bennet is forced to endure his advances and rejection, and she comes to doubt herself and her own capacities. Mrs Bennet is hurt and humiliated by her son’s behavior, while Jane is despairing and disillusioned. Collins’ actions contribute to the negative stereotypes and caricatures of women that are commonplace in Regency England.I’m not sure Elizabeth “doubts herself” or her decision: Collins rattles off a list of reasons why she will change her mind, thinking her “coy” (rather than simply repulsed). Mrs Bennet has, much to her delight I’m sure, gained a son at some point in the AI’s retelling. Jane enters the essay, either as Jane Bennet or as the author herself, smashing the fourth wall and becoming her own character. Who knows?
Collins’ role as an expression of regency era patriarchy in Pride and Prejudice has negative consequences for all of the characters in the novel. He uses his position of authority to pressure and manipulate Elizabeth, Mrs Bennet, and Jane, and his behavior contributes to the negative stereotypes and caricatures of women that are commonplace in Regency England.An accurate summary of some of the less-than-accurate points of the essay.

As you can see, the AI misses the mark on a number of important points from the text. What appears at first to be a reasonably well-written essay doesn’t really stand up to a close analysis (or even a fairly distant one). But this is a free product, the technology is developing rapidly, and at a glance this could pass for an essay written by a human.

It will become harder and harder to discern between genuine human writing and essays produced by an AI. We can’t fight the technology, nor should we. Instead, we should focus on ways to get students interacting with AI writers, testing their boundaries, correcting their mistakes, and even learning from what they have to say.

If you’re interested in reading more about how to integrate these technologies into your classroom, join the list:

Processing…
Success! You're on the list.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: