Modern And Distributed Networking
CDNs and Edge Caching
Module 1, Chapter 5 broke latency into four
JrCodex·8 min read
Jr Codex Computer Networks Notes
Level: Advanced Prerequisites: Chapter 1: Load Balancing and Reverse Proxies Time to complete: ~20 minutes
Table of Contents
- The One Thing You Cannot Optimise
- How a CDN Works
- Anycast Routing
- Cache Keys and Invalidation
- Caching Dynamic Content
- What Else a CDN Buys
- Summary & Next Steps
1. The One Thing You Cannot Optimise
The Physics
─────────────────────────────────────────
Module 1, Chapter 5 broke latency into four
components. Three can be engineered:
transmission ──► more bandwidth
queuing ──► better queue management
processing ──► faster hardware
PROPAGATION cannot. It is distance divided by the
speed of light in fibre.
London ──► Sydney ≈ 17,000 km
≈ 85 ms one way
≈ 170 ms round trip
No protocol, no hardware, no budget changes that.
─────────────────────────────────────────
Why It Dominates
─────────────────────────────────────────
Loading a page needs several round trips: DNS,
TCP, TLS, then the request (Module 5, Chapter 3).
From Sydney to a London server, that is 4 × 170ms
≈ 680ms before the first byte of content — with
infinite bandwidth.
THE ONLY FIX IS TO REDUCE THE DISTANCE.
That is what a CDN is. Everything else it does is
secondary.
─────────────────────────────────────────
2. How a CDN Works
The Structure
─────────────────────────────────────────
ORIGIN
Your actual server. One location, or a few.
EDGE / POP
Hundreds of points of presence worldwide, each
caching content.
A user is served from the NEAREST edge. The
origin is contacted only on a cache miss.
─────────────────────────────────────────
A Request, Two Ways
─────────────────────────────────────────
CACHE HIT
user ──► nearest edge (10 ms) ──► response
Total: ~10 ms.
CACHE MISS
user ──► edge (10 ms)
└► origin (170 ms) ──► edge ──► user
Total: ~190 ms, and the edge now CACHES it.
The next user in that region gets the 10 ms path.
─────────────────────────────────────────
Tiered Caching
─────────────────────────────────────────
Hundreds of edges each missing independently
would produce hundreds of identical origin
requests for one new file — a THUNDERING HERD on
every cache expiry.
A REGIONAL tier sits between edges and origin:
edges ──► regional cache ──► origin
Now one origin request serves a whole region.
Combined with request coalescing — where
concurrent misses for the same object wait on one
upstream fetch — origin load drops enormously.
─────────────────────────────────────────
3. Anycast Routing
The Mechanism
─────────────────────────────────────────
The SAME IP address is announced via BGP from
many locations (Module 3, Chapter 4).
Each router forwards toward whichever announcement
is closest by BGP's metrics. Users are routed to
a nearby edge automatically, with no DNS trickery
and no client logic.
─────────────────────────────────────────
Why It Is Elegant
─────────────────────────────────────────
AUTOMATIC PROXIMITY routing already solves
"nearest"
AUTOMATIC FAILOVER a failed site withdraws its
announcement, and traffic
reroutes in seconds
DDoS DILUTION an attack is spread across
every site rather than
concentrated (Module 6,
Chapter 4)
ONE ADDRESS no DNS-based geo-steering,
no client complexity
─────────────────────────────────────────
Its Limits
─────────────────────────────────────────
"NEAREST" IS BGP-NEAREST, NOT GEOGRAPHICALLY
NEAREST. BGP optimises for policy and AS-path
length, not distance (Module 1, Chapter 4). You
can be routed to a further site.
ROUTE CHANGES MID-CONNECTION break TCP, because
packets suddenly arrive at a different machine
with no connection state. Rare, and real.
Which is why anycast is ideal for UDP (DNS) and
used carefully for TCP — and one more reason QUIC's
connection IDs (Module 4, Chapter 5) are useful:
they survive a change of server path.
─────────────────────────────────────────
4. Cache Keys and Invalidation
The Cache Key
─────────────────────────────────────────
By default: the URL.
Two requests with the same URL are the same
object. Which is wrong when the response VARIES —
by language, by device, by encoding.
The Vary header tells the cache which REQUEST
HEADERS change the response:
Vary: Accept-Encoding, Accept-Language
─────────────────────────────────────────
THE Vary TRAP
─────────────────────────────────────────
Vary: User-Agent
There are millions of distinct User-Agent
strings. You have just created a separate cache
entry per browser version — and your hit rate
approaches zero.
Vary on LOW-CARDINALITY headers only.
Accept-Encoding has ~3 values. Accept-Language
has a few dozen. User-Agent and Cookie have
effectively infinite values.
─────────────────────────────────────────
INVALIDATION — the hard problem
─────────────────────────────────────────
PURGE explicitly delete an object from every
edge. Fast, and you must know exactly
what to purge.
SOFT PURGE mark stale rather than deleting, so it
can still be served while revalidating.
SURROGATE tag objects with keys, then purge by
KEYS tag. "Purge everything tagged
product-42" clears the product page,
the listing and the search results in
one operation.
── the technique that makes dynamic
caching manageable
TTL just wait. Simple, and slow.
─────────────────────────────────────────
The Better Answer: Do Not Invalidate
─────────────────────────────────────────
Module 5, Chapter 2's content-hashed filenames.
app.a3f9c1.js Cache-Control: max-age=31536000,
immutable
When the content changes, the URL changes. The
old object is never wrong — it is simply no
longer referenced.
Nothing to invalidate, no propagation delay, and
a perfect cache hit rate. This is why every
modern build tool hashes filenames, and it is the
single best caching decision available.
─────────────────────────────────────────
5. Caching Dynamic Content
It Is Not Only for Static Files
─────────────────────────────────────────
MICRO-CACHING
Cache a "dynamic" page for ONE SECOND.
At 1,000 requests per second, that turns 1,000
origin requests into ONE. A 99.9% reduction,
for content at most one second stale.
Extremely effective on news, product listings,
dashboards — anything read far more often than
it changes.
STALE-WHILE-REVALIDATE
Cache-Control: max-age=60,
stale-while-revalidate=300
Serve the stale copy INSTANTLY while fetching a
fresh one in the background. The user never
waits for the origin.
STALE-IF-ERROR
Serve stale content when the origin is DOWN.
Your site stays up through an origin outage —
degraded, not broken.
─────────────────────────────────────────
EDGE COMPUTE
─────────────────────────────────────────
Run code at the edge, not only cache there.
Good uses:
- A/B test assignment, personalisation of a
cached page
- authentication and authorisation checks
before hitting the origin
- request routing, rewriting, header
normalisation
- assembling a page from cached fragments
The gain is the same as caching: the round trip
to the origin is avoided. The limit is that edge
runtimes are constrained — short execution times,
limited memory, and no access to your database
unless you replicate it.
─────────────────────────────────────────
What Must NOT Be Cached
─────────────────────────────────────────
□ anything behind authentication, unless the
cache key includes the user
□ responses containing personal data
□ pages that Set-Cookie
□ anything with Cache-Control: private or
no-store
THE CLASSIC INCIDENT: a shared cache stores one
user's logged-in page and serves it to another.
It happens when a Set-Cookie response is cached,
or when the cache key ignores the session cookie.
Test this deliberately. It is a data breach, not
a bug.
─────────────────────────────────────────
6. What Else a CDN Buys
Beyond Latency
─────────────────────────────────────────
ORIGIN OFFLOAD
90-99% of requests never reach your servers.
Directly reduces the capacity you must run.
DDoS ABSORPTION
A CDN's aggregate capacity vastly exceeds any
single origin's (Module 6, Chapter 4).
TLS TERMINATION AT THE EDGE
The handshake's round trips (Module 6,
Chapter 3) happen over 10 ms rather than 170 ms
— a large saving, and it is why a CDN speeds up
even uncacheable content.
PROTOCOL UPGRADES
HTTP/3, TLS 1.3, Brotli at the edge, without
touching your origin.
AVAILABILITY
stale-if-error keeps you serving through an
origin outage.
─────────────────────────────────────────
The Point Worth Emphasising
─────────────────────────────────────────
A CDN speeds up UNCACHEABLE content too.
The connection and TLS handshakes terminate at
the nearby edge, and the edge holds a WARM,
long-lived connection to your origin — already
past slow start (Module 4, Chapter 4).
So even a pure cache-miss path is faster than the
user connecting to the origin directly. This
surprises people, and it is often the larger
share of the benefit.
─────────────────────────────────────────
# Is it being served from cache?
curl -I https://example.com/app.js | grep -iE "cache|age|x-cache|cf-cache"
# x-cache: HIT
# age: 3421 ← seconds this copy has been cached
# cache-control: max-age=31536000, immutable7. Summary & Next Steps
Key Takeaways
- Propagation delay is the one latency component nothing but reduced distance can improve, and a CDN exists to reduce it.
- Anycast routes users to the nearest edge using BGP itself, giving automatic failover and DDoS dilution — but "nearest" means BGP-nearest, not geographically nearest.
- Vary on low-cardinality headers only; varying on User-Agent creates a cache entry per browser version and destroys the hit rate.
- A CDN accelerates uncacheable content too, because the handshakes terminate nearby and the edge holds a warm connection to the origin.
Concept Check
- Why does a CDN help even for content that cannot be cached at all?
- What goes wrong with
Vary: User-Agent, and what is the general rule? - Why do content-hashed filenames remove the invalidation problem rather than solving it?
Next Chapter
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to Computer Networks Index