One early-access team deployed a hotfix, purged its CDN cache, and checked the result from several locations. The new response appeared everywhere except one of our 12 PoPs. That location continued serving the previous content long enough to look like a failed deployment, not a delayed cache update.
The purge had reached 11 of 12 PoPs. Nothing was obviously broken in the application or the deployment. The problem was that the edge had a different view of reality from the origin, and the team had no immediate way to tell whether the stale response was caused by a missed purge, a cache key mismatch, or a request that had never been cached in the first place.
That incident was not unusual. Cache invalidation at the edge is harder than it looks because caching is not one decision. It is a collection of decisions about identity, freshness, routing, failure behavior, and visibility. We have been cataloging the failure modes with our early-access teams as we build Symbiosis, where containers run at 12 PoP locations across 5 continents and the platform manages CDN-layer caching and routing.
What edge caching gets right
At its best, edge caching keeps repeated reads close to the person making the request. A cached response can avoid a trip to the origin, reduce repeated work there, and make delivery less sensitive to distance. This is especially useful for content that is identical for many users and changes on a controlled schedule.
It also gives a service more room during bursts. If many requests ask for the same cacheable object, the edge can serve those requests without making the origin reconstruct the response each time. The value is not mysterious. Fewer origin requests and shorter network paths can make a measurable difference.
Routing adds another useful property. Requests can be directed toward an available deployment location rather than following a single fixed path. But routing and caching interact. A request can reach the expected PoP while still receiving a response produced under an older cache state. A healthy route does not prove a fresh object.
Where it breaks
Freshness is not the same as deployment state
A deployment changes what the application would return now. It does not automatically remove every response that an edge may already hold. Time-based expiration eventually helps, but “eventually” is not a deployment strategy when a response contains a correction or a security-related change.
The hotfix incident exposed a common assumption: that a successful purge means every subsequent request sees the new version. In a distributed system, propagation takes time, and a request arriving during that interval can observe a different result depending on its location. With 12 PoPs, the number of places to check is manageable for a small team, but it is still large enough for a missed location to matter.
Cache keys can encode the wrong identity
A cache needs to decide which requests are equivalent. If the key omits a relevant input, two responses that should differ can be treated as the same object. If it includes too many inputs, the cache becomes fragmented and delivers fewer hits.
This is where personalized content becomes a boundary. Edge caching is good for shared, relatively stable responses. It is not automatically good for responses that depend on a user, session, authorization state, rapidly changing permissions, or an overlooked request header. Those responses need an explicit policy. “The URL is the same” is not enough to establish that the body is safe to share.
Invalidation can create its own race
Purging an object while requests are still arriving creates ordering questions. One edge may fetch the new response immediately. Another may still have the old response available. A request can also refill a cache during a deployment if the origin and edge disagree about which version is active.
There is a related failure mode in partial purges. A team may invalidate a page but not the data or asset it references, leaving a page that is new in one respect and old in another. The result can look like an application regression even though each individual cached object is behaving according to its own policy.
The less visible failures
Stale content is easy to notice because somebody sees the wrong thing. Cache poisoning and accidental sharing are more serious because the response may look valid. An overly broad cache rule, an incomplete cache key, or unexpected treatment of request metadata can cause content intended for one context to be reused in another.
Routing failures can also be mistaken for cache failures. If a request is sent to a different location, the team may compare responses from two PoPs without realizing that the request paths, deployment versions, or cache states differ. Without location-aware logs and cache status, debugging becomes guesswork.
For a small team, observability is part of the cache design rather than an optional add-on. We need to know where a response was served, whether it was a hit or miss, which policy applied, and whether an invalidation reached the relevant location. The platform can manage CDN-layer caching and routing, but no platform can remove the underlying need to define what “fresh” means for an application.
Our working boundary
We treat edge caching as a delivery optimization, not a source of truth. The origin and deployment state remain authoritative. The edge is useful when the cost of serving a slightly older shared response is understood and acceptable. It is a poor fit when correctness depends on immediate global agreement or on request context that the cache policy does not fully represent.
That boundary does not make caching simple. It makes the tradeoff visible. A longer freshness window can improve cache effectiveness while increasing the time stale content survives. A more aggressive purge can reduce that window while adding propagation work and more opportunities for partial state. A cache-bypass rule can protect correctness while sending more traffic back to the origin.
Our approach removes much of the operational work from teams that do not have an SRE dedicated to these systems, but there are still unresolved gaps. We continue to work through how to make propagation state, cache identity, and deployment changes easier to inspect without requiring every team to become an expert in distributed caching.
The lesson from the missed PoP was not that edge caching failed. It did exactly what its local state told it to do. The failure was our assumption that a distributed cache would behave like one cache. Edge delivery gets the repetitive, geographically distributed work right. It becomes dangerous when freshness, identity, and propagation are left implicit.