By Dileep solanki Ai and Data science specialist · Last updated: 12 November 2026 · Checked against the studies and sources listed below.
Editors, teachers and readers all want to know whether a piece of writing came from a chatbot. Confident answers are everywhere: a particular word, a tidy sentence, a semicolon. The research is messier. There is no reliable single tell, but it isn’t true that nobody can spot AI text. This guide covers what the evidence supports, and what a fairer process looks like.
Why detection tools can’t carry an accusation
Even OpenAI gave up on its own detector
OpenAI launched a text classifier in January 2023 and described it as not fully reliable. In its own tests it correctly flagged 26% of AI-written text and wrongly labelled human text as AI 9% of the time. By July 2023 the company noted that the classifier was no longer available due to its low rate of accuracy. OpenAI also warned that human-written text can sometimes be confidently mislabelled as AI-written, and that very predictable text can’t be reliably classified at all.
Detectors misjudge non-native English writers
A Stanford team tested seven widely used detectors on essays by native and non-native English writers. The detectors were near-perfect on US eighth-grade essays but misclassified over half of the TOEFL essays as AI-generated, with an average false positive rate of 61.22%. The problem wasn’t a few unlucky essays: all seven detectors unanimously flagged about 20% of the human-written TOEFL essays, and at least one detector flagged nearly 98%.
The researchers traced this to how many detectors work. Texts with lower “perplexity”, meaning more predictable word choice, look more machine-like, and the flagged essays did have significantly lower perplexity, suggesting detectors may penalise writers with limited linguistic expression. When ChatGPT was used to enrich the vocabulary of the same essays, the false positive rate fell from 61.22% to 11.77%.
For a publication with Indian and international readers, the implication is direct: many of your contributors write English as a second or third language, and a detector score could wrongly flag exactly those writers. That is my inference from the study, not a finding about Indian writers specifically.
Detector vendors have updated their products since 2023, and some claim much lower error rates. I could not find independent, peer-reviewed false-positive rates for the current versions of the major tools. Treat any score as a lead to investigate, not as evidence.
Can people spot AI writing instead?
Untrained readers mostly guess
A large study from the CISPA Helmholtz Center asked about 3,000 people in Germany, China and the USA to judge text, audio and image samples. Across all countries and media types, most participants classified AI-generated media as human-made. The paper reports that average detection accuracy was below 50% for images and never exceeded 60% for the other media types. Age, education and media literacy made little consistent difference.
There are important caveats. The survey ran from June to September 2022, before ChatGPT’s public launch, and the researchers generated the text and audio samples themselves. Models have improved since, so these numbers describe a snapshot, not today’s situation.
Experienced users do much better
A 2025 paper presented at ACL tested modern models (GPT-4o, Claude and o1). Annotators who frequently use LLMs for writing tasks excelled at detecting AI text, and the majority vote of five such experts misclassified only 1 of 300 articles, beating most commercial and open-source detectors. People who rarely or never use LLMs were poor detectors and overestimated their own ability.
Read this result carefully too. The five experts were top performers selected from more than 70 screened annotators. They voted as a group on American English non-fiction articles, and the study did not examine factual accuracy. They relied on specific “AI vocabulary” but also on broader qualities such as formality, originality and clarity. So a trained group assessing a whole piece can be accurate. A single reader announcing that one word or one sentence proves AI use is a very different thing.
A fairer way to check: look at the process, not the prose
If you edit or teach, the research suggests judging evidence you can verify. These are my suggestions as an editor, not findings from the studies above.
- Check the facts and sources. Do the cited studies exist, and do they say what the article claims? Invented citations and wrong details are concrete problems regardless of how the text was produced.
- Ask for process evidence. Draft history, notes or an outline are hard to fake and easy for genuine writers to supply.
- Talk to the writer. Ask about a choice or an example. Someone who wrote the piece can explain it.
- Look for first-hand detail you can verify. Specific places, numbers and experiences that check out are more telling than polish.
- Treat publishing pace as a reason to ask questions, never as proof. A very high output rate can prompt you to look more closely. But teams, ghostwriters, backlogs and legitimate AI-assisted editing can all explain it.
- Publish a clear policy. Define what is allowed (for example, AI for research or editing with disclosure) and what isn’t (unreviewed AI-generated submissions).
If you’re the writer who might be accused
- Keep version history, research notes and source links.
- Disclose how you used AI if your publisher asks.
- If you’re a non-native English writer, know that a flagged score is a known failure of the tools. Say so, and offer your drafts.
What Google says about AI-written content
For website owners, Google’s guidance is about value, not provenance. Its Search documentation says generative AI can be useful when researching a topic and adding structure to original content, but generating many pages without adding value for users may violate its spam policy on scaled content abuse. Google’s earlier statement adds that using automation, including AI, with the primary purpose of manipulating ranking violates its spam policies. In short, the risk is publishing thin pages at scale, not the use of an AI tool.
Final assessment
“Secretly using ChatGPT” is hard to prove from the text alone. Detectors have documented failures, untrained readers perform near chance, and trained readers can do well only under particular conditions. The healthier question for any editor or reader isn’t “did a bot write this?” but “is this accurate, original and honestly presented?” You can answer that with sources, drafts and conversation, and without accusing anyone on a hunch.
How this article was made
This article was researched and drafted with AI assistance and edited against the sources below. It contains no sponsored placements or affiliate links. I did not run my own detector tests. The editorial checklist and the implication for non-native writers are my own judgement, not study findings. Tools and studies change, so check for newer research. This is general information, not legal advice.
Sources and Further Reading
- Frank et al., “A Representative Study on Human Detection of Artificially Generated Media Across Countries” (CISPA): https://arxiv.org/pdf/2312.05976
- CISPA press release on the study: https://idw-online.de/en/news833930
- The Decoder, summary of the CISPA study: https://the-decoder.com/humans-can-hardly-distinguish-ai-generated-content-from-real-content/
- Russell, Karpinska and Iyyer, “People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text” (ACL 2025): https://arxiv.org/abs/2501.15654
- Full paper (v2): https://arxiv.org/html/2501.15654v2
- Liang et al., “GPT detectors are biased against non-native English writers” (Stanford): https://arxiv.org/pdf/2304.02819
- Times Higher Education coverage of the Stanford study: https://www.timeshighereducation.com/news/ai-text-detectors-biased-against-non-native-english-speakers?site=cn

