How DNS failures take sites offline without warning

The homepage still loads. You load it yourself, from the office in London, and it is completely fine, which turns out to be the first problem. Meanwhile support is filling with reports from Germany: checkout failing, then working a minute later, then failing again. Nothing is fully down. Nothing is fully up. An engineer opens the afternoon deploys, because a clean break would honestly be easier to chase than this.

What actually happened is that a DNS record changed about an hour ago. An A record was repointed during a migration, and the internet has not finished agreeing on the new answer. Some resolvers are handing out the new address. Some are still serving the old one, which points at infrastructure that is now half decommissioned, so it answers slowly or refuses the connection entirely for whoever is unlucky enough to ask it.

Why nothing looks broken

This is what makes DNS incidents frightening: they rarely fail cleanly. There is no 500 page, no flatline on a graph, no obvious moment of breakage. Half the requests are perfect. Your own uptime check resolves through a resolver that already holds the new record, connects without trouble, and reports green the entire time. The outage is real, and it is invisible from exactly where you happen to be standing.

1. 14:00A record for checkout repointed during a migration
2. 14:12Intermittent failures reported, clustered in the EU
3. 14:15Homepage loads fine from the office; uptime checks stay green
4. 14:18Engineers start reading the afternoon deploy logs
5. 14:35An unrelated deploy is rolled back. Nothing improves
6. 14:50Someone queries the record from three resolvers and gets two answers

How operators get misled

Most DNS incidents look like application instability at first. The errors are intermittent, they correlate with a region, and there is no clean cause, so the team reaches for the explanations it can act on: a bad deploy, a flaky dependency, a load balancer behaving oddly. They roll something back. Nothing improves, because the rollback never touched DNS. The time is not lost because the team is slow. It is lost because every internal signal points inward, and the cause is outside the building.

Most DNS incidents are diagnosed as application problems first and DNS problems second, usually about an hour later.

Why propagation creates partial outages

DNS has no global switch. A change spreads only as caches expire, and caches expire on their own schedule, not yours. A 300-second TTL is a suggestion: plenty of resolvers enforce their own minimums, quietly ignore low TTLs, or keep serving a record well past the point it should have let go. Negative caching means a resolver that fetched a bad answer during the change can hold onto it for minutes. And the big public resolvers are many machines wearing a single address, so two people on "the same" 8.8.8.8 can get different answers depending on which node they reached.

The result is a long, uneven tail. Resolvers disagree for longer than anyone plans for. The incident does not end when you fix the record. It ends when the slowest cache somewhere finally releases the old one, which is often well after you have already told everyone it is resolved.

What continuous monitoring actually changes

The thing that catches this is not checking whether your domain resolves. It is checking whether it resolves to the same answer everywhere, continuously, from multiple resolvers and multiple regions, and treating disagreement as the alert. A stale record that leaves one region broken while another looks perfectly healthy is precisely the signal you want surfaced, rather than averaged into a global number that erases it. Watching the record itself for unexpected edits catches the change at 14:00, before the tickets start at 14:12.

Crowswatch watches your DNS from the outside, across resolvers, and flags drift, disagreement and record changes as they happen. The point is not another dashboard. It is that a partial DNS outage shows up as a DNS event you can name, instead of twenty minutes spent suspecting a deploy that was never the problem.