I’ve spent A LOT of personal time the past year diving into AI and how it can be used to improve people’s lives. When I was getting my M.S. degree I built basic neural networks in Excel on machines with 4 Gb of memory. We’ve come a long way in the past 25 years. It’s an exciting time to be alive.
There’s no doubt that the opening of Artificial Intelligence (AI) to everyone is going to have a large impact on education. Large Language Models (LLM’s) such as Chat GPT likely have the biggest impact – both on the positive and negative sides for learning.
One large problem is students generating homework writing assignments and larger papers using LLM’s. The learning curve to using ChatGPT to generate a paper is quite low and can create good work in a short amount of time. That is making it difficult to tell who is creating using writing aids and who is not.
AI detectors were quickly created as an answer. They were designed to detect AI generated content.
They have failed. Unless you are fine with accusing innocent students of cheating, do not rely on AI detectors to catch AI generated content.
AI Detectors Are Not Reliable In Practical Scenarios
I was spurred to write this article after being alerted about a recent academic paper highlighting the problems with AI detection. On June 28, 2023 , researchers from the University of Maryland released a paper exploring the reliability of AI detectors. They found that:
In this paper, both empirically and theoretically, we show that these detectors
are not reliable in practical scenarios. Empirically, we show that paraphrasing
attacks, where a light paraphraser is applied on top of the generative text model,
can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers.
Our experiments demonstrate that retrieval-based detectors, designed to evade
paraphrasing attacks, are still vulnerable against recursive paraphrasing. We then
provide a theoretical impossibility result indicating that as language models become more sophisticated and better at emulating human text, the performance of
even the best-possible detector decreases. For a sufficiently advanced language
model seeking to imitate human text, even the best-possible detector may only
perform marginally better than a random classifier.
arXiv:2303.11156 [cs.CL]
Why is this the case?
A (Very) Short Primer For The Layperson on How LLM’s And Detectors Work
For all their hype, LLMs are at their heart a next-word prediction engine. For every given word, there’s a probability of what the next word written will be based on the training data. These type of prediction models have been around for years – what’s new is that the newest batch of LLM’s have been trained on billions or trillions of data points.
While every AI detection tool attacks the problem from a slightly different angle, most AI detectors geared towards the educational market try to compare the known probabilities of the next word from AI generated text to some pool of known human-written text. When words of low-probability are used, it’s generally considered more likely to be generated by a human.
This article by Sebastian Raschka does a great job of explaining with minimal jargon how different AI detectors work and what their faults are.
If your cheating student is exceedingly lazy, uses a generic prompt for text generation and does nothing to the resulting text – some of these tools will on average be able to catch most of the generated work.
AI Detectors Can Falsely Classify Honest Students as Cheaters
The problem is, detectors are no where near flawless. And a detector that you can’t rely on holds negative value. To their credit, higher-quality AI detectors like GPTZero plainly state that a detector should only be one of many steps in determining if a student is using AI or not – it shouldn’t be used as proof for punishment. However, that’s not the way these tools are universally being used in practice.
More generic AI detectors are even worse. There are plenty of examples online about how the US Constitition and opening chapters of the Bible all coming back as AI generated. This is due to the training set having many, many copies of famous documents – so it is often true AI would generate something nearly identical if prompted in the right way.
Falsely accusing a student of AI cheating is a surefire way to turn a child off of school. I don’t know of any teachers that would knowingly do so – but you have a chance of doing that if you rely on AI Detection tools as a standard of Truth.
Don’t do that.
Your Information Gain From AI Detectors Is Minimal
Most of the texts returned as AI generated you’ll know the text is out of the normal writing style of the student before running it through a detector. Since the AI detector isn’t anywhere near infallible, it’s only real purpose is to use as a tool to get a bluffing student to admit they used AI.
It’s also very rare that a tool will return a 100% or 0% chance of text being AI generated. What are you to do as a teacher if it returns a 35% chance, but you are suspicious of a student for other reasons? A 50% chance? A 79% chance but the text is in the style your student normally uses for writing and you have no other reason to suspect cheating? Are you ready to face accusations of bias, or explain how you can accuse someone you know is using AI yet “the AI detector” says chances are under 50%?
Having information that you can’t rely on 100% isn’t really helpful – especially in a world where the consequences of a false accusation of cheating can bring lawyers into the picture pretty quick.
So If AI Detectors Don’t Work – How Do I As A Teacher Deal With Student AI Writing
Here’s the problem: In most US classrooms, you will have trouble catching average and above students using AI as the system is currently set up.
If the work is sent home to be completed, a determined student that wants to use AI will have multiple ways to do so. The more time they are willing to put into cheating, the less easy it will be to detect it.
I already heard a 12 year old boy talk about how he can defeat AI detectors. He simply rewrites the “AI first draft” (as he called it) in his own words, and hands that in. There’s not a lot you can do about that – except if the AI hallucinates false facts that the student doesn’t catch.
But there are steps you can take for less determined miscreants…
- Show Your Work: The idea here is that it will take just as much time to cheat and get away with it as it would to write the paper by themselves. This can take a few forms – the more you ask for the harder it becomes to use AI to save time. You can ask to see the history of revisions in Google docs. You can ask for notes, prior drafts, and actual copies of sources where available. The side benefit is if a student actually does cheat and goes through all these steps, they likely still learned the material. They are just not getting the benefit of honing their own writing skill.
- Control The Writing Environment: You can make writing an in-school activity where using AI tools are easily detectable. This may require a rethink of what the best use of school time is for and moving to a new learning model – such as a flipped classroom. Considering AI isn’t going away, it’s a opportune time to begin this process anyway.
- Q&A After Paper Submission: In the end, the most important outcome is the student has learned the material. Asking some paper related questions after submission is one way to see if the student actually learned the hoped-for outcome.
- AI Can’t Do Everything: In a similar vein as the above idea, come up with questions only the student can answers. Relate the topic to their own lives and have them answer it in a personal way.
There are probably quite a few other actions you can take that I haven’t heard of yet – add them in the comments below if you have some good ideas.
Prepare For The Future That Is Already Here
Now that anyone can spin up their own LLM for under $100, it’s clear that AI is here to stay. Educators will need to deal with the very real threat of cheating – especially in writing tasks.
But it’s important that we don’t over-react and derails a child’s love of learning through a false accusation relying on a failing AI detector. While AI detectors seem at first glance to be a convenient tool for catching cheaters, in practice they aren’t helpful because you can’t predict how and which way they will fail.
Instead of relying heavily on imperfect tools, use this opportunity to change how you teach. That will allow you to avoid the hamster-wheel of trying to stay up with the daily changes in AI text generation, and allow you to do what you do best – teach.