Operational Intelligence
Incident Analysiswarning

Why third-party outages are harder to detect than you think

The hardest outages to catch are the ones you do not run. Why dependency failures hide, and how to surface them.

15 May 20267 min read

Your dashboards stay green. Support tickets start arriving, almost all of them from Germany. Engineers open the deploy logs, because nothing internal appears broken and the last release is the obvious suspect. They find nothing. Twenty minutes later, Cloudflare posts "Investigating elevated edge latency in EU-West." The incident had been live the entire time. It just was not yours.

This is the defining quality of a third-party outage: every system you own is behaving correctly. Your code is fine. Your servers are fine. Your tests pass. The thing that failed sits outside the boundary of everything you monitor, and most monitoring is built to watch the inside of that boundary very closely and the outside not at all.

Why your monitoring cannot see it

Your checks run from somewhere, usually a single region close to your infrastructure. A provider degrading at the edge in EU-West does not fail a synthetic check running from us-east-1. Your error rates stay nominal because requests that reach your origin succeed; the ones that never make it past a struggling edge node never become requests you can count. The only authoritative signal lives on the provider, and the provider has not told you yet.

The regional blindfold

Most dashboards report a global average, and a global average is the single most effective way to hide a regional outage from the people responsible for it. EU customers are abandoning checkout while your aggregate success rate moves from 99.4 percent to 99.1 percent: a number that would not wake anyone. The outage is real and total for one market and statistically invisible to you.

  1. 14:31Edge latency begins rising for EU-West traffic
  2. 14:33First support tickets arrive, clustered in Germany
  3. 14:34Engineers check the latest deploy and internal logs. Clean
  4. 14:41Someone opens the provider status page. Nothing posted yet
  5. 14:51Provider posts "investigating elevated edge latency, EU-West"
  6. 15:10Status moves to "identified"

The twenty minutes between 14:31 and 14:51 is the expensive part. It is spent investigating yourself: rereading a clean diff, restarting healthy services, asking whoever deployed last to explain a change that was never the problem. None of that work was wrong given what the team could see. It was just aimed at the wrong layer because the right layer was invisible.

The hardest outages to detect are the ones you did not cause and cannot fix.

How dependencies hide

You depend on far more than the architecture diagram shows. The CDN, the DNS provider, the payment API, the email relay, the auth provider, the object store, the package registry your deploy pulls from at the worst possible moment. Most of them are invisible precisely because they normally work, so they never earn a place on a dashboard. And several of them have their own dependencies, so a failure two providers upstream reaches you through a chain nobody has ever drawn.

What changes when you watch the ecosystem

The fix is not more internal monitoring. You can instrument your own systems perfectly and still spend twenty minutes blind to a Cloudflare incident. The change is to watch the providers you depend on directly, pulling their status the moment they publish it, and to correlate that with your own regional signals. When "EU checkout latency up" and "provider edge incident in EU-West" surface as one connected event instead of two unrelated graphs in two different tools, the twenty minutes of investigating yourself simply does not happen. You start the incident already knowing it is not yours to fix, which is the most useful thing you can know in the first five minutes.

Crowswatch watches the providers, domains and dependencies behind signals like these, and connects them into one operational view.

Monitor your dependencies with Crowswatch

More operational briefings