Can I consistently detect unedited AI-generated prose? Let’s experiment.
Early studies show that some people can reliably distinguish AI-generated from human-authored text. That aligns with my experience. I believe I can distinguish unedited AI prose with a high rate of accuracy, and I want to test that belief.
To that end, I recently asked my LinkedIn followers to post short passages so I could guess whether they were human- or AI-written, then identify some of the heuristics I used to make that guess. The shorter the passage, the harder it is to distinguish AI prose from human prose, so this experiment poses a tough test. But it forces me to articulate some of the indicators I see of AI-generated prose so I can share them in future posts.
I fear that frequent exposure to AI-generated text will affect humans’ writing styles, since our styles constantly evolve in response to what we read. The best corrective is to read as much high-quality prose (especially literature) as you can while being especially selective about your online reading diet. Another is to learn the patterns characteristic of AI-generated text.
Put me to the test:
Post a passage in the comments that you know with certainty is either human-written or AI-generated. Please follow these rules:
125–160 words
Self-contained and understandable without surrounding context
Prose (not poetry, dialogue, etc.)
Not highly technical
No citations, quotations, or references
Not edited to seem more human- or AI-like
Not translated from another language or rewritten from another source
If human-written, written without AI assistance or editing
I’ll be looking mainly at stylistic choices such as diction and syntax, not checking for hallucinations.
I’ll reply with:
Human or AI
My confidence level
The specific features or heuristics that influenced my judgment
Then tell me whether I was right. If the passage was AI-generated, please identify the exact model, and tell me whether you used unusual prompting, supplied writing samples, requested a particular style, or otherwise did anything beyond supplying an ordinary prompt.
Warning: If you wrote the passage, submit it only if you have a thick skin. This exercise requires me to give candid, critical feedback on the passages submitted.
Why I pay attention to AI’s writing tells
“Fable Likes an Idea,” by Steve Yegge (hat tip: Zvi Mowshowitz)
Like most people, I prefer human-produced art and authentic communication, so I have no interest in reading anyone’s AI-generated text. I use AI for all sorts of purposes — coding productivity and self-education projects,1 planning family outings, deciding which headphones to buy, proofreading briefs, etc. — but not to write. Using AI to write robs you of the benefits of the writing process, which include learning the topic you’re writing about, generating your own ideas, and acquiring a sense of investment in the work.
In everyday life and when teaching, we should err on the side of trusting that a human wrote a text unless there’s significant evidence to the contrary. But when it comes to deciding which Substack blogs, LinkedIn posts, or other online ephemera I’ll devote my limited reading time to, I have to be ruthless: if I suspect that what I’m reading might be slop, I unsubscribe and unfollow.
Please share this post with your networks:
Subscribe for free to receive future Elegant Legal Writing posts by email:
Ryan McCarl is a partner of the business litigation firm Rushing McCarl LLP and author of Elegant Legal Writing (Univ. Cal. Press 2024). For more tips about legal writing and argumentation, take one of Ryan’s online writing workshops, subscribe to the Elegant Legal Writing blog, and follow Ryan on LinkedIn. McCarl’s book is available in print, e-book, and audio formats.
Connect with Ryan McCarl: Order the Elegant Legal Writing book | LinkedIn | Rushing McCarl LLP | Linktree | Second Stage (blog) | Bluesky | Substack | Twitter (X) | Threads | Mastodon
Recent projects include a private language-learning podcast, a distraction-free app for email and Slack, and a simplified replica of my retired language-learning app WordBrewery. Claude Code — which can be used for far more than coding — has made my Python skills and familiarity with command-line workflows feel like superpowers.