How to Detect AI Use (in Texts)

You probably know that artificial intelligence is gaining ground every day: there doesn’t seem to be a single industry immune to the effect programs like ChatGPT or its alternatives have. It could be that you’re very worried that coworkers, employees or even your superiors are using AI of some kind to cut some corners on their work, or that some of your students are trying to pass off GPT’s work as their own. Let’s go over some of the ways you can detect AI use.

Using AI Detection Tools

When faced with a technological dilemma, the first place to turn is to more technology: with the rise of AI we have seen an equally fast rise of AI detection software. Tools like Copyleaks and Sapling claim to be able to detect AI use in any text you feed it — and even will tell you in percentages how much of a given text is human or AI made.

At first glance, these tools are pretty impressive: feed it a text and it will come back telling you if it’s human-made or not. We tested the free tool Content@Scale for a bit, and texts that were made by us were passed, while AI-generated ones (which we copy-pasted wholesale) were rejected.

So far so good, you’d think, but looking closely you can start to see some issues. For one, it seemed that the detection software went section by section, figuring out what was made by humans and what wasn’t. It’s not just this tool that does this, almost all work in some variation of this method.

Pattern Recognition

Thing is, the software seems to be looking for patterns, and that is where you run into issues. When speaking or writing, all of us have patterns. Even seasoned writers have a tendency to describe similar things using more or less the same words. The good ones just don’t let it pop up more than once in a single article.

However, some patterns are more common than others, and this is where AI detection software runs into issues.

Using the same text analysis as before, you can see that this text was flagged as maybe being AI generated, likely because it seemed similar to text the tool has in the database. In this case, because it’s surrounded by original text, it didn’t affect the final rating. However, were this to happen more than once, there’s a good chance the tool may throw up a positive for AI-generated content.

Patching Up Patterns

This is the crux of the issue. As the example above shows, it’s recommending we change the text up a bit. However, there are only so many ways in which we can say things before they turn into gibberish. The nature of language is that of patterns; when we don’t adhere to them it’s either very bad, or very good — and then it’s called poetry.

This is an issue that can affect any writer, but it’s generally worse for those starting out. When you’re still working on developing your skills — a process that’s never finished, but that’s a discussion for another time — most people have a tendency to fall back on familiar patterns. Generally speaking, “younger” writers will use more cliches than more experienced one, who will avoid them like the plague (*coughs*).

The result is that most AI detection software throws up a lot of positives when checking texts, especially work done by less experienced writers. Veterans aren’t exempt from getting flagged, but it does seem to affect newer writers more. As such, they’re quicker to fall afoul of accusations of using AI, harming them and the publications they work for. So what can you do if you can’t trust software?

Using Humans to Detect AI Use

Though software tools have their place — that percentage score can help confirm any hints of suspicion — the best way to detect AI use is probably to use your own judgment. Naturally, there are ways to go about this: just because one writer’s article is a bit wooden doesn’t mean they’re using AI, maybe they were having a bad day.

There are a few things to look out for to see whether an article was written with aid of an AI. First up is the wooden delivery mentioned before. When reading a “pure” AI article you can’t help but realize that this wasn’t touched by human hand, it just doesn’t work. Sure, the differences between GPT-3.5 and GPT-4 are noticeable, but they’re nowhere near human.

While generative AI and other large language models can do their best to mimic human speech and writing, they always seem to miss the mark — and that’s likely something that won’t change even if GPT-5 comes about. Just look at the example below:

Though this snippet isn’t entirely without merit, there are a lot of issues with this text. Aside from the obnoxious use of adjectives and adverbs, it just doesn’t gel right. The child who isn’t like anybody else, the tingling in veins; it’s all stuff right out of every other substandard fantasy story you’ve ever read.

How AI Builds Content

In a way, this is the clue we’re looking for and the way we can determine whether something was made by AI: as a language model, generative AI can only work from pre-existing content. It then amalgamates it and, when prompted, spits out that amalgam in the way you prompted it to. Thing is, though, that as a computer it’s best suited to storing and reproducing information more or less in the form of a checklist.

When you prompt an AI, that’s what you get, a list of stuff, even if it doesn’t at first glance look like one. Take a second glance, though, and you’ll quickly catch on: in our prompt above, there’s a boy unlike anybody else, check, from an idyllic village, check, who goes out in search of adventure, check.

Note that while this creates bad writing, it’s actually a great way to use AI to improve your writing. The built-in checklist makes it easy to see if you missed something, and as a grammar and spelling checker it is a great tool for writers, especially those just starting out.

From this follows that the best way to detect AI use is to look for these kinds of formulaic structures that feature a lot of adjectives and other, not-quite natural turns of phrase. If a text is supposed to be fictional, be on the lookout for overused tropes and tired turns of phrase. When looking for academic or professional use of AI, look for texts that feel copy-pasted or like a listicle.

Can You Detect AI Use?

Of course, this advice assumes that people are using AI and only AI, without editing it some. This is where things get tricky: with some ingenuity, you can disguise AI content and make it seem more natural. It still won’t be very good and the effort involved is roughly on par with writing something from scratch, but there are still people who will go to these lengths.

One good way around AI detection is to only use AI for sections of an article, so prompt a few things, but write other sections yourself. Though we’re not sure how this works as a labor-saving device, it would be a good way to get past anybody looking for signs of AI use.

There are simpler tricks, too, like just editing the text like you would any other, but then adding a few typos here and there. Though programs like GPT are far from flawless, they don’t make mistakes like that. Judicious use of typos and the occasional grammar SNAFU may throw less sophisticated detection methods off.

However, whether you use a program to find out AI use or try to do it yourself, the fact of the matter is that you won’t always be able to find it, and you may want to wonder whether it’s even worth the effort. The quality of AI is about on par with a very poor writer, so you may want to think long and deep about why you’d want that level of writing done in the first place.