CAP theorem and PACELC

Think of it like this

Analogy

Two bank branches share one customer ledger by phone. One afternoon the phone line goes dead. A customer walks into branch A and wants to withdraw money.

• Branch A can refuse until the line is back, so the balance can't be wrong. That's choosing consistency.
• Or it can pay out and sort it out later, risking the customer withdrawing the same money at branch B too. That's choosing availability.

There's no third option where it pays out and is guaranteed to be correct. That's the CAP theorem.

The problem it solves

Once data is replicated across machines (lesson 11), those machines talk over a network, and networks fail. Links drop, switches misbehave, a data center loses its connection. When that happens, replicas can't confirm they agree. You need to decide in advance what your system does then, or it will decide for you, usually badly.

A real example

Example

An ATM that loses its connection

A storm cuts the line between an ATM and the bank's central systems. That's a network partition. Someone wants to withdraw $100.

If the bank chooses consistency, the ATM refuses: it can't check the real balance, so it won't risk a wrong answer. If it chooses availability, the ATM pays out and records the withdrawal to sync later, accepting that the account might end up overdrawn.

Many banks pick a middle path: allow small offline withdrawals up to a limit and reconcile when the line is back. CAP doesn't choose for you; it makes you decide ahead of time what to do when the network splits.

How it works

The CAP theorem names three properties:

• Consistency (C): every read sees the latest write, as if there were one copy (strong consistency, more precisely called linearizability).
• Availability (A): every request to a working node gets a non-error response.
• Partition tolerance (P): the system keeps working when messages between nodes are lost or delayed.

A network partition is not optional in real systems, so the theorem says: during a partition, you must give up either C or A.

Try it. Make a few writes, cut the network, write to one side and read from the other, then heal it. Do it once in CP mode and once in AP mode.

CP: refuse rather than be wrong

A CP system rejects requests (or the side without a majority does) while it can't confirm the latest data. Users see errors or timeouts, but never wrong data. Examples: systems built on consensus such as etcd, ZooKeeper and Spanner; a relational database with synchronous replication; anything handling money, inventory counts or locks.

AP: answer anyway, fix it later

An AP system keeps answering on both sides. Data may be stale and the two sides may accept conflicting writes, which are reconciled when the network heals (last-write-wins, merging, or asking the user). Examples: DNS, shopping carts, social media likes, Cassandra and DynamoDB in their default modes. This leads to eventual consistency (lesson 21).

PACELC: the trade-off when things are fine

CAP only talks about partitions, which are rare. PACELC adds what happens the rest of the time:

If there is a Partition, choose Availability or Consistency; Else, choose Latency or Consistency.

Even with a healthy network, a strongly consistent write must wait for replicas, possibly in other regions, to confirm. That's slower. A system that writes to one replica and replicates in the background is faster but can serve stale reads.

System (default settings)	During partition	Normally
DynamoDB, Cassandra	Available	Lower latency
Spanner, CockroachDB	Consistent	Consistent (pays latency)
MongoDB (majority writes)	Consistent	Consistent
Cosmos DB	Depends on the chosen consistency level	Depends

Many databases let you choose per request, for example strongly consistent reads at a higher latency and cost.

Choosing in practice

Ask what's worse for this piece of data: an error, or a wrong answer?

• Bank balances, seat bookings, unique usernames, distributed locks: a wrong answer is worse, so CP.
• Likes, view counts, feeds, product catalogs, shopping carts: an error is worse, so AP, with reconciliation.

A single product usually mixes both: payments CP, the activity feed AP.

Trade-offs

What you gain

• CP: data is never contradictory, which makes the app simpler to reason about.
• AP: the system stays up and fast, even across regions and during failures.
• Making the choice explicit per feature avoids surprises during outages.

What it costs you

• CP: errors and higher latency during partitions and across regions.
• AP: stale reads and conflicts; the app must cope with reconciliation.
• CAP is a simplification. Real systems offer many in-between consistency levels.

When not to use it

• Don't use CAP to label a whole database as one thing forever; many are tunable per request.
• Don't over-think it for a single-node database with no replicas: there's no partition between copies to worry about.

In the cloud

Replicated and multi-region databasesEvery major cloud sells this as a managed service, so you configure it rather than build it.

AWS2

• Amazon Aurora Global Database (opens vendor docs)

  One primary region with read-only copies in other regions, usually under a second behind.

• Amazon DynamoDB global tables (opens vendor docs)

  Multi-region, multi-writer tables; replicas converge with last-writer-wins.

Azure2

• Azure Cosmos DB (multi-region) (opens vendor docs)

  Replicate to any region, optionally write everywhere, and pick one of five consistency levels.

• Azure SQL Database active geo-replication (opens vendor docs)

  Readable secondary databases in other regions with manual or automatic failover.

Google Cloud2

• Spanner (opens vendor docs)

  Globally distributed relational database with strong consistency across regions.

• Cloud SQL cross-region read replicas (opens vendor docs)

  Read-only copies of a Cloud SQL instance in another region, promotable on disaster.

Writes go to a leader; replicas in other zones or regions serve reads and take over on failure.

AWSAzureGoogle Cloud

1. App
2. Aurora writer
3. Aurora readers
4. Secondary region (Global Database)

Interview questions

Try answering out loud first, then open the model answer.

Explain the CAP theorem in your own words.+

Model answer

In a distributed system where data is copied across machines, the network between them can fail. When it does, a node that can't reach the others has two options for a request: answer with what it has, which keeps it available but may be stale or conflict with the other side, or refuse, which keeps it consistent but unavailable. You can't have both during that partition. When the network is healthy you can have both. PACELC adds that, even then, stronger consistency costs latency.

For a ticket-booking system, would you choose CP or AP?+

Model answer

For seat inventory, CP: selling the same seat twice is worse than asking a user to retry. I'd use a strongly consistent store or a transaction with a short hold on the seat. Other parts can be AP: the event catalog, seat-map display (with a final check at checkout), and recommendations can be served from caches and replicas that might be slightly stale.

How do AP systems resolve conflicting writes after a partition heals?+

Model answer

Common approaches: last-write-wins using timestamps (simple, but silently drops one write and depends on clocks); version vectors to detect true conflicts and keep both versions for the app or user to merge; CRDTs, data types like counters and sets designed to merge automatically; and application-specific rules, for example a cart that takes the union of items. The choice depends on whether losing a write is acceptable.

Quiz

Pick an answer to see whether it’s right and why.

1. 01According to CAP, what's the real choice a distributed system makes?

   Pick any two of consistency, availability and partition tolerance, all the timeDuring a network partition, choose between consistency and availabilityChoose between SQL and NoSQL
2. 02A shopping cart that should always accept "add to cart", even if it occasionally shows an item twice, is leaning towards…

   CPAPNeither
3. 03What does the "ELC" part of PACELC say?

   Even with no partition, there's a trade-off between latency and consistencyEncryption lowers consistencyEventual consistency is always lower latency than strong consistency during partitions
4. 04During a partition, a system chooses consistency. What do some users see?

   Stale dataErrors or timeouts instead of possibly outdated answersFaster responses
5. 05Why isn't “CA” a real choice for a system spread across machines?

   Network partitions will happen, so you must decide what to do when they doCA systems are too expensiveCA needs special hardware
6. 06A “like” counter loses contact with part of the cluster. What's usually the better choice?

   Consistency: reject likes until it healsShut down the whole siteAvailability: accept likes and merge the counts later

0 of 6 answered

Key terms

CAP theorem

During a network partition, a distributed system must choose between consistency and availability.

Network partition

A network failure that splits machines into groups that can't reach each other.

PACELC

Extends CAP. If Partitioned, choose Availability or Consistency; Else, choose Latency or Consistency.

Strong consistency

Every read sees the most recent write, as if there were only one copy of the data.

Eventual consistency

Copies may disagree for a while, but if writes stop, they all end up the same.

Replication

Keeping copies of the same data on more than one machine.