Over the summer, I began seeing the first suspected cases of AI use in the introductory college writing courses that I teach online. Since then, AI-generated essays have become a more common element of these classes.
Thankfully, I’ve gotten much better at instantly spotting AI papers thanks to some tell-tale signs of ChatGPT’s and other AI-generators’ writing styles. I discuss the techniques I’ve learned to detect AI writing in my classes below.
Before we get to these strategies, however, it’s important to remember that suspected AI use isn’t immediate grounds for disciplinary action. These cases should be used as conversation starters with students and even – forgive the cliché – as a teachable moment to explain the problems with using AI-generated work.
To that end, I’ve written previously about how I handled these suspected AI cases, the troubling limitations and discriminatory tendencies of existing AI detectors, and about what happens when educators incorrectly accuse students of using AI.
With those caveats firmly in place, here are the signs I look for to detect AI use from my students.
1. How to Detect AI Writing: The Submission is Too Long
When an assignment asks students for one paragraph and a student turns in more than a page, my spidey sense goes off.
Almost every class does have one overachieving student who will do this without AI, but that student usually sends 14 emails the first week and submits every assignment early, and most importantly, while too long, their assignment is often truly well written. A student who suddenly overproduces raises a red flag.
2. The Answer Misses The Mark While Also Being Too Long
Being long in and of itself isn’t enough to identify AI use, but it’s often overlong assignments that have additional strange features that can make it suspicious.
For instance, the assignment might be four times the required length yet doesn’t include the required citations or cover page. Or it goes on and on about something related to the topic but doesn’t quite get at the specifics of the actual question asked.
3. AI Writing is Emotionless Even When Describing Emotions
If ChatGPT was a musician it would be Kenny G or Muzak. As it stands now, AI writing is the equivalent of verbal smooth jazz or grey noise. ChatGPT, for instance, has this very peppy positive vibe that somehow doesn’t convey actual emotion.
One assignment I have asks students to reflect on important memories or favorite hobbies. You immediately sense the hollowness of ChatGPT’s response to this kind of prompt. For example, I just told ChatGPT I loved skateboarding as a kid and asked it for an essay describing that. Here’s how ChatGPT started:
As a kid, there was nothing more exhilarating than the feeling of cruising on my skateboard. The rhythmic sound of wheels against pavement, the wind rushing through my hair, and the freedom to explore the world on four wheels – skateboarding was not just a hobby; it was a source of unbridled joy.
You get the point. It’s like an extended elevator jazz sax solo but with words.
4. Excessive Use of Lists and Bullet Points
Here are some reasons that I suspect students are using AI if their papers have many lists or bullet points:
1. ChatGPT and other AI generators frequently present information in list form even though human authors generally know that’s not an effective way to write an essay.
2. Most human writers will not inherently write this way, especially new writers who often struggle with organizing information.
3. While lists can be a good way to organize information, presenting more complex ideas in this manner can be .…
4 … annoying.
5. Do you see what I mean?
6. (Yes, I know, it’s ironic that I’m complaining about this here given that this story is also a list.)
5. It’s Mistake-Free
I’ve criticized ChatGPT’s writing here yet in fairness it does produce very clean prose that is, on average, more error-free than what is submitted by many of my students. Even experienced writers miss commas, have long and awkward sentences, and make little mistakes – which is why we have editors. ChatGPT’s writing isn’t too “perfect” but it’s too clean.
6. The Writing Doesn’t Match The Student’s Other Work
Writing instructors know this inherently and have long been on the lookout for changes in voice that could be an indicator that a student is plagiarizing work.
AI writing doesn’t really change that. When a student submits new work that is wildly different from previous work, or when their discussion board comments are riddled with errors not found in their formal assignments, it’s time to take a closer look.
7. Something Is Just . . . Off
The boundaries between these different AI writing tells blur together and sometimes it’s a combination of a few things that gets me to suspect a piece of writing. Other times it’s harder to tell what is off about the writing, and I just get the sense that a human didn’t do the work in front of me.
I’ve learned to trust these gut instincts to a point. When confronted with these more subtle cases, I will often ask a fellow instructor or my department chair to take a quick look (I eliminate identifying student information when necessary). Getting a second opinion helps ensure I’ve not gone down a paranoid “my students are all robots and nothing I read is real” rabbit hole. Once a colleague agrees something is likely up, I’m comfortable going forward with my AI hypothesis based on suspicion alone, in part, because as mentioned previously, I use suspected cases of AI as conversation starters rather than to make accusations.
Again, it is difficult to prove students are using AI and accusing them of doing so is problematic. Even ChatGPT knows that. When I asked it why it is bad to accuse students of using AI to write papers, the chatbot answered: “Accusing students of using AI without proper evidence or understanding can be problematic for several reasons.”
Then it launched into a list.