Writers have been stung by critics for as long as there’s been writing. Samuel Pepys declared Shakespeare’s A Midsummer Night’s Dream ‘the most insipid ridiculous play that ever I saw in my life’. The plot of Aristophanes’ play, The Frogs (405 BC), asserts that the only way to find a good poet back then was to ‘bring a dead one back from Hades’. And a writing board from the Egyptian Middle Kingdom (ca. 1981–1802 BC) shows a teacher’s red marks correcting a student’s ‘many spelling mistakes’.

But a new form of critique emerged not long after the arrival of ChatGPT in November 2022: the accusation that a writer’s work was created not by a human at all but by a machine. Today, there’s an arsenal of tools to help people search for signs of AI-generated text (AIGT), including Brandwell, Copyleaks, GPT-2 Detector, GPTZero, Grammarly, Monica, Originality AI, Quillbot, Undetectable AI, Winston AI, Writer and ZeroGPT.

You can find reports that claim some of these tools are 100 per cent accurate, but that’s hardly a consensus view. Even AI companies in the business of promoting AI writing assistants acknowledge these tools often stumble, either by flagging human-written content as AI-generated or by mistakenly assuring users that AI-generated content wasn’t written by a bot. Researchers have also found that AI detection tools show bias against writers who aren’t native English speakers.

After running 60 documents of known origin through eight different AIGT detection tools in September, Business Insider attached this disclaimer to its findings: “We recommend using AI detectors with caution, and [Soheil] Feizi [founder and CEO of the startup RELAI and assistant professor in computer studies and director of Reliable AI Lab at the University of Maryland] dissuades readers from using detectors at all. Detector results alone should not be used to make significant decisions about academic honesty, hiring or job status.”

 

A look inside the black box

So how do these AIGT detection tools work, and what are their potential points of failure?

Researchers generally recognise four types of detector tools: watermark-based, neural network-based, zero-shot and retrieval based.

AI models that use watermarking encode a signal of some kind into the content that they generate, which detectors can then identify algorithmically. But these have yet to prove reliable and can be manipulated. For example, shortly after it released ChatGPT, OpenAI began offering a watermark-based tool to look for signs of content written with ChatGPT, but it pulled the tool after only six months for ‘its low rate of accuracy’.

The other types of detectors are trained on two types of large datasets – human-written and AI-generated – and use AI to identify pattern differences between the two. But the accuracy of such tools can depend on a lot of factors: the AI model used to generate the content in question, whether a combination of models was used, whether AI-generated content was partially or completely revised by a human, and whether AI-generated content was altered by so-called humanisers, which are AI tools that use algorithms to revise text to reduce signs of AI authorship.

And then there’s the fact that AI text generators and AI text detection tools are locked in an endless arms race: as text generation tools are updated to improve their ability to put out more human-sounding content, detectors must be retrained and fine-tuned to update their detection capabilities. As Sider AI notes: “This is the Red Queen problem: detectors must run faster just to stay in place.”

All of these elements combined make a strong case for being cautious and sceptical when using AI detection tools. There are simply too many variables in how – and how well – they work to use their outputs to accuse human writers of not writing their own words.

“AIGT detection tools are not accurate and reliable to use,” concluded one team of researchers who reviewed multiple studies about detector accuracy. “The extent of AI detectors’ accuracy is more effective on older generative AI models, wherein they start to lose their accuracy and reliability in their detection once newer generative AI models are used.”

Their advice? “It should be taken into consideration that AI text detectors can be incorrect, so it must be used with caution and preferably alongside human reviewers. The repercussions of the results provided by the detectors must be kept in mind as they might cause irreversible damage to the authors’ reputation as well as the institutions.”