jon@stjohn
← All posts

CockroachDB Multi-Region: Picking the Right Survival Goal

· 2 min read · CockroachDBDistributed Systems

One of the first design decisions you’ll make when deploying CockroachDB across multiple regions is choosing a survival goal. It sounds like a small setting, but it quietly determines how your cluster behaves during a region outage — and how much latency you pay for that guarantee on every write.

The two survival goals

CockroachDB gives you two choices at the database level:

1
2
ALTER DATABASE app_db SURVIVE ZONE FAILURE;
ALTER DATABASE app_db SURVIVE REGION FAILURE;
  • Zone failure — the default. Your cluster tolerates the loss of an availability zone within a region without downtime or data loss. It does not guarantee survival if an entire region goes offline.
  • Region failure — the cluster tolerates the loss of an entire region. This requires a minimum of three regions and replicates data across all of them, which means writes need agreement from replicas outside the local region more often.

Why this isn’t a free upgrade

It’s tempting to reach for SURVIVE REGION FAILURE everywhere — after all, more resilience sounds strictly better. In practice, the Raft consensus protocol underlying CockroachDB has to place replicas so that losing any one region still leaves a quorum. That placement is what costs you:

1
2
3
4
5
6
7
-- Regional by row table: each row's leaseholder lives in its "home" region
CREATE TABLE orders (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  customer_id UUID NOT NULL,
  region crdb_internal_region NOT NULL,
  total_cents INT NOT NULL
) LOCALITY REGIONAL BY ROW;

With REGIONAL BY ROW, reads and writes for a given row are fast when they originate in that row’s home region, because the leaseholder is local. Cross- region traffic — a customer in us-east reading a row homed in eu-west — pays a round trip.

A rule of thumb

  • Start with SURVIVE ZONE FAILURE unless you have a concrete compliance or uptime requirement that demands surviving a full region outage.
  • If you do need region failure tolerance, pair it with REGIONAL BY ROW tables and make sure your application routes users to their home region wherever possible — the survival goal protects you from disaster, not from bad routing.
  • Measure p99 write latency before and after switching survival goals in a staging environment that mirrors your real region topology. The difference is often larger than teams expect.

Multi-region is one of the areas where CockroachDB gives you real knobs instead of a single “cloud native” checkbox — it’s worth understanding what each one actually costs before you turn it on in production.