Ethics Law And Trust
Provenance and Watermarking
These are routinely conflated and they solve different problems with different reliability.
JrCodex·8 min read
Jr Codex Generative AI Notes
Level: Intermediate Prerequisites: Chapter 2: Deepfakes, Consent & Misuse Time to complete: ~20 minutes
Table of Contents
- Three Different Mechanisms
- C2PA — Signed Content Credentials
- Invisible Watermarking
- Text Watermarking
- Why Detection Alone Fails
- What to Actually Implement
- Summary & Next Steps
1. Three Different Mechanisms
These are routinely conflated and they solve different problems with different reliability.
The Three
─────────────────────────────────────────
METADATA / C2PA
A cryptographically signed record travelling with
the file describing how it was made.
Strong claim, easy to strip.
WATERMARKING
A signal embedded in the CONTENT itself.
Weaker claim, harder to strip.
DETECTION
A classifier guessing after the fact whether
content is synthetic.
No claim at all — a probability.
─────────────────────────────────────────
The Key Distinction
─────────────────────────────────────────
Metadata and watermarks are added BY THE PRODUCER
and are therefore only as trustworthy as the
producer's participation.
Detection needs no cooperation and is therefore
the only option against a hostile producer — which
is exactly why it is the weakest (Section 5).
─────────────────────────────────────────
2. C2PA — Signed Content Credentials
C2PA (Coalition for Content Provenance and Authenticity) is the emerging cross-industry standard, backed by camera manufacturers, software vendors and model providers.
What a Manifest Records
─────────────────────────────────────────
- What captured or generated this (device or model)
- When and, optionally, where
- What EDITS were applied, in order
- Which earlier asset it was derived from
- A cryptographic signature over all of the above
Chained: each edit adds a new signed entry, so the
manifest is a verifiable EDIT HISTORY, not a single
claim.
─────────────────────────────────────────
The Important Reframe
─────────────────────────────────────────
C2PA does not detect fakes. It ASSERTS
AUTHENTICITY.
A camera signs "I captured this, unedited, at this
time." An editor signs "I cropped it." A model signs
"I generated this."
This is the direct answer to the liar's dividend
(Chapter 2): it lets REAL content prove itself,
which detection can never do.
─────────────────────────────────────────
from c2pa import Reader, Builder
def attach_credentials(in_path, out_path, model, model_version, signer):
"""Sign a generated asset at the moment it is produced."""
manifest = {
"claim_generator": "jrcodex-studio/1.0",
"assertions": [
{"label": "c2pa.actions", "data": {"actions": [
{"action": "c2pa.created", "digitalSourceType":
"http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia"},
]}},
{"label": "com.jrcodex.model", "data": {"name": model, "version": model_version}},
],
}
Builder(manifest).sign_file(in_path, out_path, signer)
def verify(path):
"""Read credentials back. Returns None when there are none."""
try:
return Reader.from_file(path).json()
except Exception:
return None # absent OR stripped — you CANNOT tell which (see below)The digitalSourceType value is the part that matters: it is the standardised vocabulary term for "produced by a trained algorithm," which is what makes the disclosure machine-readable rather than a free-text note.
The Honest Limitation
─────────────────────────────────────────
Metadata is trivially STRIPPED. A screenshot, a
re-encode, or any platform that discards metadata
on upload removes it entirely.
So absence of a manifest means NOTHING — it is not
evidence of anything. Only PRESENCE of a valid
signature carries information.
Read it as a one-directional signal: it can confirm,
never deny.
─────────────────────────────────────────
3. Invisible Watermarking
Where metadata is a label on the outside, a watermark is embedded in the content, surviving operations that destroy metadata.
How Image Watermarking Works
─────────────────────────────────────────
A pattern is embedded across the image in a domain
chosen to be robust — typically frequency components
rather than raw pixels.
Requirements in tension:
IMPERCEPTIBLE no visible change
ROBUST survives crop, resize, re-encode,
screenshot, moderate colour shifts
SECURE not removable without knowing the
key
You cannot maximise all three. Every scheme picks a
point on that trade-off.
─────────────────────────────────────────
Where It Is Applied Matters
─────────────────────────────────────────
POST-HOC applied to the finished image. Simple,
and removable by anyone who knows to
look for it.
IN-MODEL baked into the generation process — for
a latent diffusion model, into the VAE
DECODER itself (Module 3, Chapter 1),
so every image it produces carries the
mark inherently.
In-model is substantially more robust, because there
is no separable "watermarking step" to skip.
─────────────────────────────────────────
Robustness, Honestly
─────────────────────────────────────────
SURVIVES JPEG re-compression, resizing, moderate
cropping, screenshots, colour adjustment
DOES NOT heavy crops, adversarial removal
attacks, regeneration through another
generative model, aggressive
re-styling
A determined, informed adversary can remove any
watermark. That is not a reason to skip it: it stops
CASUAL misuse and unintentional laundering, which is
the large majority of real cases.
─────────────────────────────────────────
4. Text Watermarking
Text is the hardest modality, and knowing why is instructive.
Why Text Resists Watermarking
─────────────────────────────────────────
An image has millions of values with enormous
redundancy — plenty of room to hide a signal
imperceptibly.
A paragraph has a few hundred tokens and NO
redundancy. Every token is semantically load-bearing.
There is nowhere to hide anything.
─────────────────────────────────────────
The Green-List Approach
─────────────────────────────────────────
At each generation step, hash the previous token to
pseudo-randomly split the vocabulary into a "green"
list and a "red" list. Add a small bias to the green
logits (Module 2, Chapter 2's logit_bias mechanism).
Detection: recompute the lists over a candidate text
and count green tokens. Natural text hits ~50%.
Watermarked text hits significantly more, and the
gap is statistically testable.
─────────────────────────────────────────
Its Limits
─────────────────────────────────────────
- Needs a few hundred tokens for statistical
confidence; a tweet cannot be watermarked
- PARAPHRASING destroys it almost entirely, and
paraphrasing is one API call
- Biasing the vocabulary at all is a small quality
cost, which providers are reluctant to pay
- Requires the generating provider's cooperation
Net: text watermarking is real, deployed in places,
and not something to build a policy on.
─────────────────────────────────────────
5. Why Detection Alone Fails
Post-hoc classifiers that judge whether content is AI-generated are widely marketed and structurally unreliable.
The Structural Problems
─────────────────────────────────────────
MOVING TARGET a detector trained on today's models
degrades against next quarter's. The
generator improves faster than the
detector.
BASE RATES at 95% accuracy on a corpus that is
1% synthetic, MOST positives are
FALSE. The maths does not care how
good the classifier feels.
ADVERSARIAL light editing, paraphrase, or a pass
through another model defeats most
detectors.
UNEQUAL HARM text detectors demonstrably
misclassify non-native English
writing as AI-generated at higher
rates. Deploying one in an academic
or hiring context therefore
discriminates.
─────────────────────────────────────────
The Resulting Rule
─────────────────────────────────────────
NEVER take a consequential action against a person
based on a detector's output alone.
Not an academic penalty. Not a hiring rejection. Not
an account ban.
Detection is a signal for investigation. It is not
evidence.
─────────────────────────────────────────
6. What to Actually Implement
If You GENERATE Content
─────────────────────────────────────────
1. Attach C2PA credentials at generation. Libraries
exist; the cost is small.
2. Use a model with in-model watermarking where
available.
3. Log what you generated, for whom, and when —
your own records are the most reliable
provenance you control.
4. Disclose synthesis visibly wherever a viewer
could otherwise be deceived.
If You CONSUME Content
─────────────────────────────────────────
1. Verify C2PA where present, and treat a valid
signature as meaningful.
2. Treat ABSENCE as no information at all.
3. Never act against a person on a detector's
output alone.
4. For anything consequential, verify OUT OF BAND —
contact the purported source directly. This is
the only method that does not degrade as models
improve.
The Direction of Travel
─────────────────────────────────────────
The industry is converging on PROVING AUTHENTICITY
rather than DETECTING SYNTHESIS — signed capture at
the camera, signed edits through the pipeline,
verification at display.
It is the right direction because it is the only one
that gets STRONGER as generation improves, rather
than weaker.
─────────────────────────────────────────
7. Summary & Next Steps
Key Takeaways
- Metadata, watermarking and detection are three different mechanisms: signed provenance is a strong but strippable claim, watermarks are weaker but survive more, detection is only a probability.
- C2PA asserts authenticity rather than detecting fakes, which is why absence of a manifest carries no information while presence of a valid signature does.
- In-model watermarking — embedded in the VAE decoder — is far more robust than a post-hoc step, though no watermark survives a determined adversary.
- Detectors fail structurally on base rates, adversarial edits and unequal error rates, so their output is grounds for investigation and never for action against a person.
Concept Check
- Why is the absence of a C2PA manifest not evidence that content is synthetic?
- Explain why text is fundamentally harder to watermark than images.
- A 95%-accurate detector flags a student's essay in a corpus that is 1% AI-written. Why is acting on that flag indefensible?
Next Chapter
→ Chapter 4: Bias, Safety Filters & Responsible Deployment
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Generative AI Index