Multi-region HA on AWS and GCP: the boring parts that matter

Multi-region high availability is the single most over-built infrastructure pattern we are asked to deliver. Most teams start a multi-region project the day they hit 10,000 users, which is roughly the day they have earned the right to a single well-tuned region with backups. Here is how we decide, and what we build when the answer is yes.

You probably do not need multi-region yet

Multi-AZ (multiple availability zones within one region) gives you 99.95% availability, which is 4.4 hours of downtime per year. The remaining 0.05% comes from regional events: natural disasters, cloud provider mistakes, fiber cuts. Most startups will hit product-market fit and Series B before they lose a regional event. Build multi-AZ first; you can almost always add multi-region later without rearchitecting if you have kept your state stores portable.

When you do, RPO and RTO come first

Recovery Point Objective is how much data you can afford to lose. Recovery Time Objective is how long you can be down. Without numbers, you cannot size the architecture. We ask clients to write these down before we write a line of Terraform. A team that cannot commit to numbers like '5 minute RPO, 30 minute RTO' is not ready for multi-region.

The DNS problem nobody talks about

Failover at the DNS level has TTL constraints. A 60-second TTL means your regional failover can take up to 60 seconds to be visible to clients. A 300-second TTL means clients keep hitting the dead region for five minutes after you flipped. We usually combine Route 53 / Cloud DNS health-check failover with a load balancer that drains in-region traffic, giving us both speed and safety.

Data residency is the hidden cost

Multi-region means your data lives in two places. That has GDPR implications, SOC 2 implications, and backup implications. The first multi-region we build for a regulated client takes 40% longer than the second one, because the second one has the data classification and residency framework already in place.

DR runbooks that you actually run

The disaster recovery runbook is a document nobody reads. We make ours executable: a single command that fails over the primary database, promotes the standby, flips the DNS, and alerts the on-call. We run it quarterly in a real region we can lose without consequence. The first run always surfaces a problem nobody predicted.

IaC versus clickops

If you cannot rebuild your infrastructure from a single terraform apply, you do not have multi-region. You have two regions of which one you are afraid to touch. Multi-region in clickops is the most expensive way to lose data.

Cost ceilings in the IaC itself

We put monthly cost estimates in the Terraform output and break CI when a change increases them above a threshold. Most surprise cloud bills are surprise because nobody had a number that would have flagged them. A ceiling is the cheapest guardrail you will install.