Skip to main content

Multi-region and multi-continent

By the end of this page you will know how to home a table in a region so its writes commit at local quorum latency, what a cross-region transaction still costs, how every region reads locally, what a degraded placement looks like, and which topology to pick for your workload.

The headline: writes stay local when the table is homed​

A transaction that touches only tables homed in its own region commits at region-local quorum latency, the same latency you would get from a single-region cluster. It never pays a wide-area round trip. That is the point of homing a table: pick the region its writes happen in, and its write path never leaves that region's network.

Every write still goes through the same synchronous-quorum consensus mechanism described in Multi-zone, at any distance; that has not changed and is not a mode you can turn off. What changed is who sits in a table's quorum. Left unhomed, a table's voters are ranked across the whole cluster the same way they always were. Home a table in a region, and its voters are ranked only among that region's live members, so the quorum a write needs to reach never crosses a border it does not have to.

For a business with European data sovereignty requirements, this is not just a latency optimization: home a table in eu-central, and every voter holding its data, and every write's quorum, stays inside that region, provably, by construction.

Labelling nodes: region and zone​

Give each node a region, and optionally a finer zone within it, in [cluster]:

[cluster]
region = "eu-central"
zone = "eu-central-1a"

Both default to unset. region is the one that matters for placement: when a node has it set, a table homed in that region keeps all its voters on nodes labelled with it, so its transactions commit at region-local quorum latency instead of WAN. zone is a finer failure domain inside the region (rack or AZ); it is reported alongside region, but see Multi-zone for what it is actually used for.

Both are replicated cluster state, not local configuration only one node knows about: every node's own region and zone are visible cluster-wide.

SELECT node_id, region, zone, role, state FROM scram.nodes;

Leave both unset and a node behaves exactly as it always has; a cluster with no region configured anywhere places every table exactly as it did before this feature existed, byte-identical.

Homing a table​

Home a table at CREATE TABLE time with WITH (home_region = '...'):

CREATE TABLE eu_customers (
    id BIGINT PRIMARY KEY,
    name TEXT,
    country TEXT
) WITH (home_region = 'eu-central');

Every voter for eu_customers' buckets is now ranked only among nodes labelled region = "eu-central". A transaction that only touches eu_customers, and other tables homed the same way, commits at that region's own quorum latency.

Re-home an existing table with ALTER TABLE ... SET:

ALTER TABLE eu_customers SET (home_region = 'us-east');

The move is live and asynchronous: the statement returns once the new home is durably recorded, and the actual voters migrate to the new region in the background without taking the table offline. scram.shards.pending_owners shows exactly which buckets are still converging:

SELECT "table", bucket, replicas, pending_owners
FROM scram.shards
WHERE "table" = 'eu_customers';

An empty pending_owners means the move already completed.

For a table that is read far more than it is written, and needs to stay writable from anywhere rather than commit fast from one region, spread its voters across every region instead of homing them in one, with the keyword GLOBAL:

CREATE TABLE product_catalog (
    sku TEXT PRIMARY KEY,
    name TEXT,
    price NUMERIC
) WITH (home_region = 'GLOBAL');

Syntax and case rules, exactly as the engine enforces them:

  • home_region takes a quoted string, or the bare keyword GLOBAL.
  • GLOBAL is matched case-insensitively: 'global', 'Global', and 'GLOBAL' all mean "spread voters across every region," and resolve internally to no home region at all, the identical placement an unhomed table gets on a regionless cluster.
  • Any other value is a literal region name, matched case-sensitively, byte-for-byte, against the region your nodes are labelled with. 'eu-central' and 'EU-Central' are different home regions; a typo that matches no node's label is legal SQL, but no node ranks as in-region for it.
  • An empty string is refused, loudly: 'home_region' must not be empty; omit it to use the creating node's own region, or set it to 'GLOBAL' to spread voters across regions.
  • Omit home_region entirely on CREATE TABLE and the table homes in the creating node's own region automatically: create it through a node labelled region = "eu-central" and it homes there with zero DDL changes. On a regionless cluster (no node has region set), omitting it keeps a table exactly as unhomed as it always was.
  • ALTER TABLE ... SET (...) accepts only the home_region key in this form; any other key in the parentheses is refused, naming it, rather than silently ignored. Re-homing also requires cluster mode and a table that already has shard buckets (a PRIMARY KEY table that has been assigned a shard map); a table with neither is refused rather than silently accepted.

Homing narrows a table's own write quorum to one region. It does not touch group0, the cluster's shared metadata group that every DDL statement and every timestamp batch passes through: that group still spans however many regions hold a voter, so schema changes stay priced at whatever WAN distance separates them, exactly as described below in The latency floor.

What a cross-region transaction costs​

Homing does not make a cross-region transaction a special case or forbid it, it makes it exactly what it looks like: a transaction that writes to tables homed in two different regions runs ordinary two-phase commit between those regions' quorums. It is correct, unconditionally, the same guarantee every ScramDB transaction gets. It is also priced at WAN latency for the round trip between the two homes, because that is what committing across two independent quorums actually requires, and there is no way to make that free without giving up correctness.

Design your homing around this honestly: put the tables one transaction writes together in the same home region, and that transaction is cheap. Split them across regions, and that transaction pays the trip.

Local reads, everywhere​

A region that is not a table's home can still read it locally. Add a learner-role node (see Scaling: Learner nodes) labelled with that region:

[cluster]
node_name = "node-ap-1"
region = "ap-southeast"
zone = "ap-southeast-1a"
learner = true

A learner replicates every table in the cluster, including ones homed in a different region, and never joins a write quorum. Once it has caught up, reads in its region can be served from it instead of crossing to the table's home region: opt in per-session with the learner_read and max_staleness settings, which bound exactly how stale a local answer is allowed to be before the read is refused rather than silently served anyway.

The degraded flag: when a region can't fully staff its own table​

Homing a table in a region asks for every voter to live there. If that region does not currently have enough live members to supply the table's replication factor, ScramDB does not refuse to place the table, and it does not silently serve a smaller quorum either: it fills the shortfall from outside the region, using the same deterministic ranking, so the table still gets a full quorum. That quorum is now degraded: at least one voter sits outside the home region, and that bucket's commits pay cross-region latency until the region has enough members again.

This is never silent. The moment a table is placed or re-homed this way, the server log carries a warning naming the table, its home region, and how many members were actually available there against the replication factor asked for. Check current placement yourself, any time:

SELECT "table", bucket, home_region, replicas
FROM scram.shards
WHERE home_region <> '';
SELECT node_id, region FROM scram.nodes;

Compare the two: a bucket whose replicas includes a node whose region does not match the table's home_region is a degraded bucket, paying WAN on every write to it until the region is staffed back up to its replication factor.

The latency floor​

A write that touches a shard group needs a quorum of that group's replicas to acknowledge it before it commits, which includes any DDL, since that goes through the cluster's metadata group too. The commit's latency floor is bounded below by the round trip to whichever replica needed for that majority is slowest to respond. That is ordinary networking physics, not a ScramDB measurement: cross-continent round trips commonly run 60 to 150 milliseconds or more one-way depending on the specific city pair, and that is added, unavoidably, to every write whose quorum spans the continents involved.

Retune the election timeouts for the distance​

The default randomized election window (1 to 2 seconds) and the 50 millisecond heartbeat are tuned for same-region latency. Left at the default across a WAN link whose round trip approaches or exceeds that window, ordinary network jitter can trigger spurious leader elections, the same failure mode the default is already known to be sensitive to even at same-region latencies under load, just easier to trigger over a slower or higher-jitter link.

Guidance, not a fixed number: raise election_timeout_min and election_timeout_max (and proportionally heartbeat_interval) whenever a shard group's replicas span a WAN link with round trips approaching or exceeding the default window. There is no single correct replacement value; the right number is a function of the measured round-trip time between your actual sites, so measure it and size the window from that, rather than copying a number from documentation.

A worked two-region topology​

The repository ships a runnable two-region example: two voters in eu, two in us, one cluster.

Kubernetes cannot vary an environment variable per pod inside a single StatefulSet, so a per-region deployment is one StatefulSet per region. The base StatefulSet becomes the eu pool, patched to 2 replicas and REGION=eu; a second one, scramdb-us, is the us pool. Both share the base's headless Service and ConfigMap, so every pod of both pools resolves through the same per-pod DNS and joins the same 4-node cluster.

The shared config template renders each pod's [cluster] block from its own REGION/ZONE environment variables at container start (a sed substitution in the container command, the same rendering pattern the other reference overlays use):

[cluster]
node_name = "__NODE_NAME__"
advertise_addr = "__NODE_NAME__.scramdb-headless:7190"
region = "__REGION__"
zone = "__ZONE__"
bootstrap_expect = 3
discovery = ["dns", "swim"]
dns_name = "scramdb-headless"
replication_factor = 2

bootstrap_expect = 3 counts the other voters this node waits for; with 2 pods in each of two pools, that is 4 nodes total, 3 other voters from any one node's perspective. replication_factor = 2 is deliberate: with exactly two voters in each region, a table homed in either region is placed entirely from that region's own pool, never degraded, because two live members are always enough to supply a replication factor of two.

Bring it up the same way as the other reference overlays (see Forming a cluster):

kubectl apply -k k8s/overlays/regions-2x2

Create a table through an eu pod and it homes in eu with no DDL change at all. Re-home it live with ALTER TABLE ... SET (home_region = 'us'), and scram.shards.pending_owners shows the two-voter handoff to the us pool while it converges.

What to place where​

Three topologies cover most cases:

Single region, multiple AZs. The well-supported, low-latency-cost way to get HA. This is the right default answer for most readers; see Multi-zone.

Multi-region within a continent (for example three regions on the same coast, round trips roughly 10 to 40 milliseconds). Home each region's own tables to itself, and the cross-region cost only applies to the transactions that actually need to cross regions; a table's own writes stay at that region's local quorum latency. Workable with the default consensus timeouts in many cases.

True multi-continent, whether that is one table's home region spanning oceans by its own voters' placement, or a GLOBAL table spread across every region. Nothing in the engine forbids this, and it is a legitimate, supportable topology, but every write to a GLOBAL table, or to a table whose home region a client is far from, pays the full cross-ocean round trip. This suits a workload whose write-latency budget already tolerates 100 milliseconds or more, such as a system of record with infrequent, latency-tolerant writes and a genuine need for global durability. It does not suit a low-latency OLTP workload; for that, home the table where it is actually written, one region at a time.

Next​

  • Multi-zone if a single region's AZs already meet your fault-tolerance needs.
  • Failover for what a client observes during an election, at any distance.
  • Scaling for adding and removing nodes from a running cluster.