LLM Watermarking - A Dead End

Original Here

Introduction

The European Union’s AI Act has transformed the detection of AI-generated content from a theoretical challenge into a rigid regulatory mandate. As of August 2, 2026, providers are required to label outputs in a machine-readable format, with the Commission pushing for unproven technical fixes like watermarks and cryptographic provenance. However, this top-down directive imposes significant technical burdens on developers while largely ignoring the practical impossibility of implementing these requirements effectively across diverse mediums.

While the goal of curbing academic dishonesty and fraud is noble, forcing compliance through invisible signatures is a simplistic solution to a complex problem. By prioritizing administrative transparency over technical reality, the EU risks creating a system that is easily bypassed by bad actors while penalizing legitimate providers with costly, quality-degrading requirements that fail to provide the absolute certainty regulators envision.

In some ways we already have informal versions of this. Certain habits have become associated with AI writing. The em dash has probably never received so much suspicious attention. So have certain constructions—“It’s not X, it’s Y”—along with excessive headings, strangely balanced lists and a growing collection of words which have become stereotypically “ChatGPT.” None of these, of course, prove anything. Humans used em dashes before ChatGPT and will presumably continue using them after.

A mandatory technical watermark would attempt to replace these unreliable stylistic hunches with something far more systematic: an invisible signal deliberately inserted into the output by the model itself.

The problem is that this is considerably easier to imagine than to implement.

We already know how to hide information inside photographs, video and audio through a family of techniques broadly known as steganography. Text, however, is a fundamentally different medium. A photograph contains millions of values which can be changed slightly without a human noticing. A sentence contains comparatively few choices, and changing any one of them can alter its meaning or quality.

That difference makes reliable watermarking of LLM outputs much harder than it initially appears - and raises the possibility that mandating it may create a system which is simultaneously easy to defeat and costly to the quality of the models it is supposed to regulate.

Steganography

Steganography is the practice of hiding information inside something else.

Consider a digital photograph. A photograph contains millions of pixels, each represented by numerical values describing things like red, green, and blue intensity. Those numbers contain an enormous amount of information.

Suppose the red value of a particular pixel is 184. Changing it to 185 would almost certainly be invisible to a human looking at the image. Do this across thousands of pixels and you can encode a surprising amount of information without noticeably changing the photograph.

One simple scheme could treat even numbers as a 0 and odd numbers as a 1. By adjusting selected pixel values by one, a computer could hide a binary message throughout the picture.

184 becomes 0.
185 becomes 1.

Repeat this across enough pixels and you can encode an identifier, timestamp, cryptographic signature, or some other information indicating where the image came from.

Video offers even more possibilities because a video contains thousands of images displayed sequentially, along with audio. There are therefore enormous numbers of values which can be manipulated slightly without a viewer noticing.

There are substantial problems with mandatory steganography in photographs and videos as well, particularly once media is compressed, cropped, filtered, resized, screenshotted, or deliberately manipulated. But I will leave those problems to the side.

The important point is that images contain an enormous amount of entropy. There are millions of little numerical choices available to whoever creates the image, and many of those choices can be altered without meaningfully changing what a human sees.
Text does not offer the same luxury.

The Text Problem

Imagine asking ChatGPT:
What is the capital of Burkina Faso?
The correct answer is:
Ouagadougou.

There is nowhere to hide anything.
ChatGPT cannot change “Ouagadougou” to “Timbuktu” because it needs to answer the question correctly. It cannot change the capitalization without potentially looking strange. It cannot add three unnecessary paragraphs merely because it needs enough space to insert a watermark.

The same problem exists with mathematics.
What is 7 × 8?
56.

There simply isn’t enough freedom in the answer to encode meaningful hidden information.

This produces an immediate limitation on any universal text-watermarking scheme. Short answers, factual answers, numbers, names, translations of fixed phrases, and many other outputs contain too little entropy to reliably carry a hidden message.
So short responses are effectively impossible to watermark in this way.

What about longer writing?
This becomes more interesting.

Word Baskets

The most plausible method would probably involve what we might think of as baskets of words. Suppose an AI is writing the following sentence: The proposal would increase economic growth.
There are several words the model might reasonably have used:

increase
boost
raise
improve
accelerate

An AI system normally assigns probabilities to all of these possible next words. Perhaps “increase” has a 30% probability, “boost” 25%, “raise” 20%, and so on.
A watermarking system could secretly divide possible words into two baskets.

Basket 0 might contain:
increase
raise
Basket 1 might contain:
boost
improve
accelerate

If the hidden message requires the next encoded bit to be 0, the model would be pushed toward words in Basket 0. If the hidden bit is 1, it would be pushed toward Basket 1.

Choosing one word rather than another would serve approximately the same function as changing a pixel value from 184 to 185.

Over hundreds or thousands of words, these tiny choices could accumulate into a detectable statistical pattern. The watermark would therefore not necessarily appear as a literal hidden sentence. Instead, someone examining the text could ask whether the model chose words from one predetermined group substantially more often than would occur naturally.

This is probably the most important distinction between image and text watermarking. With an image, we can make tiny changes to numerical values that are essentially irrelevant to the image’s meaning. With text, almost every change potentially affects meaning, tone, rhythm, clarity, or style.

That creates the following tradeoff: the harder you push the model to preserve the watermark, the more you interfere with its ability to produce the best answer.

Watermarks Have a Cost

Imagine that the best possible next word is “increase,” but the watermark requires a word from the other basket. Now the model chooses “boost.” That probably does not matter. But repeat the process thousands of times. Sometimes the second-best word will be almost indistinguishable from the first. Sometimes it will be noticeably worse. Sometimes changing the word will require restructuring the sentence. Sometimes the alternative will subtly change the meaning.

Over a long enough document, the watermark becomes a constraint on the model. The AI is no longer simply asking: What sequence of words best answers the user’s question? It is asking: What sequence of words best answers the question while also satisfying this hidden encoding requirement? Those are not identical objectives.

The stronger and more recoverable the watermark, the more likely it is to degrade the quality of the output. This problem becomes especially obvious with computer code.

Watermarking Code

Natural language contains a lot of redundancy. There may be twenty reasonable ways to express approximately the same idea.
Code is much less forgiving. A programmer does not necessarily want the model selecting between function names, variable structures, library calls, formatting conventions, or algorithms because those choices help preserve some invisible watermark.
Sometimes there are many equivalent ways to write a program. Sometimes there are not.

A token-level watermarking system could force the model away from the most efficient, readable, secure, or conventional implementation simply because another valid token helps preserve the hidden statistical pattern. In normal prose this might produce an awkward sentence. In code it could produce a bug.
Even when the program still works, the watermark could interfere with performance, maintainability, security, or compatibility.
This is an odd bargain. We would intentionally make AI-generated code worse so that we could later determine that it was AI-generated.

Watermark Removers

There is another, larger problem. Even if text watermarking works, the person trying to defeat it knows what the defender is trying to accomplish.

Watermark removal in images generally works by disrupting the hidden signal while preserving the visible content. An image can be compressed, resized, cropped, blurred slightly, passed through a filter, regenerated, or otherwise modified. Each transformation potentially destroys some of the carefully manipulated values carrying the watermark. The analogous attack on text is even easier to understand. Rewrite it.

If an AI generates a 1,000-word essay containing a statistical watermark, another model could be instructed: “Rewrite this while preserving the meaning.” A secondary model would execute countless new grammatical and lexical choices, altering words, shifting sentences, merging paragraphs, and eliminating transitions. As a result, the initial statistical footprint risk being severely compromised or entirely erased. We already have tools marketed as AI “humanizers” whose entire purpose is to do approximately this.

This means the second layer of evasion would not be restricted to sophisticated computer scientists. Imagine a student cheating on an assignment. Today, perhaps the student copies the ChatGPT output directly into the submission box. Under a mandatory watermarking regime, the student adds one additional step:
ChatGPT → Humanizer → Submission. Many already do.

The system may catch the least sophisticated cheaters and plagiarizers, namely the people who copy and paste AI output without modification. Anyone who understands the detection system would simply use a secondary tool to rewrite it.
There is an arms-race problem here. Once a watermarking system becomes consequential, there is an economic incentive to build tools which remove it.

And because text carries so little redundant information compared with images, destroying the watermark would be considerably easier than inserting it robustly in the first place.

The Larger Mistake

There is also a more fundamental problem with the entire project.
We are spending enormous effort trying to preserve an academic and professional world in which we pretend machine co-writers do not exist. They do and they are going to get better.
Students will use them. Professors will use them. Lawyers will use them. Programmers will use them. Government officials will use them.

Trying to construct an invisible technical tripwire inside every AI-generated paragraph may therefore be attacking the wrong problem.

Instead, we should treat machine co-writers as a given.
I discussed this broader idea in my Cyborg Scholars piece. The relevant question is increasingly not whether a human received assistance from a machine. It is whether the human understands, evaluates, improves, and takes responsibility for the resulting work.

The European Union’s mandate for AI content watermarking represents a misguided regulatory attempt to solve complex social and academic challenges with flawed technical interventions. Because text inherently lacks the high entropy of images, embedding invisible signals necessitates compromising the quality, efficiency, and utility of AI-generated content, particularly in technical domains like code. Moreover, such watermarks are easily rendered ineffective by simple rephrasing, turning detection into an endless and futile arms race. Instead of policing the use of machine co-writers through superficial tripwires, regulators and institutions should focus on fostering human accountability, critical evaluation, and responsible collaboration with AI tools.