In the era of AI, what does it mean for a document to be fake?

Real vs fake seems like it should be a bright line. Something is either real or it is not. However, with AI, the line turns into more of a gray area.

Claude just announced that it will be watermarking text it generates. That watermark persists even if the text is copied and pasted. Light editing of the text may also keep the watermark.

One commenter called this a good thing, because it can be used to identify what is “fake.” But is it really that clear? Let’s imagine that I wrote the first draft of this article myself. I then give it to Claude to review, edit, and improve my writing. Claude then takes one of my paragraphs and rewrites it to make it clearer. In doing so, Claude doesn’t change the meaning or message of my paragraph, nor does it change any facts I reference. Now, that paragraph is watermarked by Claude. If you copy and paste it into Claude, it will tell you that Claude wrote it. Is that fake? Does it take away from my message?

Let’s take it a step further – what if the document in question is relevant to a legal proceeding? Does it matter whether it was written by a human, by AI, or a combination of the two? It certainly depends on the situation. Origin matters a lot when we are talking about intellectual property rights, but likely doesn’t matter at all if we are talking about a contract where both sides signed the final agreement.

What about when we are talking about admissibility in court? An outside document is generally hearsay (there are certainly some exceptions). Hearsay requires a declarant – a person who said or wrote those words out of court. Machines are not declarants. If a document is flagged as AI because of its watermark, does that mean it is not hearsay, since a human did not write it? Maybe we need to rethink hearsay in the age of AI. AI is a new kind of machine – it’s not the same as previously machine-generated text that provided predictable outcomes.

How about if a student uses Claude to edit their paper? Claude then suggests some grammar changes and improves wording or clarity in some paragraphs. The student still conducted all the research and developed their own viewpoints. What they used Claude for is just a better version of what Word can do for spelling and grammar. A teacher then runs the paper through Claude, and it says it was AI-generated. Did that student cheat? Is the paper “fake?”

What if this watermarking starts occurring when you use a writing assistant tool like Grammarly/Superhuman Go? No one would think using this kind of tool results in “fake” text, yet it may not be that different than using Claude.

This gray area gets complicated very quickly.

I’m not opposed to watermarks. They can be immensely useful if someone is trying to use AI to create deepfakes, falsify images or videos, or pass off truly fake data as real. But the issue becomes more complex as you consider the gray areas and the potential for harm.