Cloud system design interview: a framework and a worked example

What the cloud system design round tests

A general system design interview asks whether you can assemble load balancers, caches, queues and databases into something that serves the requirement. A cloud system design interview asks the same question, then adds the constraints of real infrastructure. The design has to live somewhere, fail somewhere, authenticate to something and appear on a bill.

In practice, that means four things come up that a generic round can skip:

• Failure domains. Hosts, availability zones and regions fail independently and at different rates. The interviewer wants to know which of them your design survives, and what the user sees when it doesn't.
• Managed-service choice. Every component is a build-or-buy decision. A managed database removes patching and failover work but brings its own limits, quotas and failure behaviour. You should be able to say what you are giving up.
• Identity and boundaries. Which workload can call which, with what credentials, across which network path. Security is part of the architecture, not a slide at the end.
• Cost. Running three zones, a standby region and cross-region replication has a price, and data transfer is often the line item candidates forget.

The framework, step by step

Interviewers reward candidates who are predictable in a good way. Run the same nine steps every time. You will not spend equal time on each, and the interviewer will pull you off the sequence, but having it means you never forget security or cost because the conversation went somewhere interesting.

1. Requirements and service-level objectives

Split requirements into functional (what the system does) and non-functional (how well it must do it). Then turn the non-functional ones into numbers you can design against:

• Availability target, expressed as an SLO on a specific indicator, such as "99.9% of API requests succeed over 30 days". Saying "highly available" commits you to nothing.
• Latency target at a percentile, such as p99 under 300 ms, measured at the edge or at the service.
• Durability and recovery: how much data the business can lose (recovery point objective, RPO) and how long it can be down (recovery time objective, RTO) in a disaster.
• Constraints: data residency, compliance scope, an existing provider, a team that has never run Kubernetes.

Ask which of these the interviewer cares about most. The answer tells you where the deep dive will go.

2. Back-of-the-envelope estimates

Estimates exist to make decisions, not to show arithmetic. Do just enough to answer: does this fit on one database, how many instances, how much storage per year, and is bandwidth a cost problem? State every assumption so the interviewer can correct it.

Assumptions (stated, not known)
  10 M daily active users × 20 requests each   = 200 M requests/day
  Average: 200 M / 86,400 s                     ≈ 2,300 req/s
  Peak-to-average ratio of 3                    ≈ 7,000 req/s at peak
  10% of requests are writes                    ≈ 700 writes/s at peak

Storage
  20 M writes/day × 2 KB per record             ≈ 40 GB/day
  × 365                                         ≈ 15 TB/year, before replicas and indexes

Compute (one instance sustains ~500 req/s at target utilisation)
  7,000 / 500                                   = 14 instances at peak
  Survive the loss of one of three zones:
  the remaining two zones must carry 14         → 7 per zone, 21 in total

Egress
  7,000 req/s × 10 KB average response          ≈ 70 MB/s ≈ 560 Mbit/s at peak

The useful outputs here are not the numbers themselves. They are the conclusions: 700 writes a second fits comfortably on a single well-provisioned relational primary; 15 TB a year means you need a retention and archival story; and designing for zone loss costs you 50% more compute than the peak alone, which is a number worth saying out loud.

3. APIs and access patterns

Sketch the two or three most important operations, then describe how data is read and written: read-to-write ratio, key distribution, query shapes, consistency needs per operation. Access patterns choose your data store far more reliably than preference does. "Fetch an order by ID" and "list a customer's orders by date" are different patterns, and a key-value store handles the first well and the second only if you design the key for it.

4. Data stores and consistency

For each piece of state, decide where it lives, how it is replicated, and what consistency the reader gets. Useful questions to answer aloud:

• Is this the system of record, or a derived copy (cache, search index, analytics table) that can be rebuilt?
• Does any operation need a read-your-writes guarantee, or a transaction across rows?
• Is replication across zones synchronous (no loss on zone failure, some write latency) and across regions asynchronous (fast writes, some loss on regional failure)? That split is the common default, and you should know why.
• What is the partition key, and which key gets hot first?

5. Placement across zones and regions

Draw the failure domains before you draw the boxes. Stateless tiers spread across at least two zones, usually three, behind a load balancer that health-checks per zone. Stateful tiers need an explicit answer for zone loss. Then decide on regions: one region with multi-zone resilience is the right answer for many services, and you should say so. A second region is justified by an RTO or RPO the single region cannot meet, by data residency, or by latency for users far away. It is not justified by "more is safer".

6. Scaling and back-pressure

Explain how each tier scales and, more importantly, what happens when it can't scale fast enough. Autoscaling reacts in minutes; a traffic spike arrives in seconds. The design needs a way to shed or delay load: queues between tiers so producers aren't blocked by slow consumers, rate limits per tenant, timeouts shorter than the caller's timeout, and retries with exponential backoff and jitter so a recovering dependency isn't flattened by a retry storm. Name what gets dropped first when the system is saturated.

7. Security and identity

Cover the four questions an interviewer will otherwise ask for you: how users authenticate, how workloads authenticate to each other and to cloud APIs (short-lived credentials from workload identity, never long-lived keys in config), where the network boundaries are (private subnets, private endpoints to managed services, controlled egress), and how data is encrypted and who controls the keys. Keep it to a minute unless the interviewer wants more.

8. Cost

Identify the two or three largest cost drivers and one lever for each. Typical candidates: compute held for zone-loss headroom, a warm standby region, cross-zone and cross-region data transfer, and storage that grows forever. Express cost per unit where you can, such as cost per thousand requests or per active customer, because that is how a business decides whether a design is affordable.

9. How the design evolves

Finish with what you would change at ten times the load, and what you deliberately left out. This is where you show judgement: the first version doesn't shard the database, doesn't run active-active, and says exactly which signal would trigger either change.

A 45-minute plan

Most design rounds run between 45 and 60 minutes. Use this plan as a default and let the interviewer redirect you. If they skip ahead to a deep dive, follow them and pick up the missed steps briefly at the end.

Worked example: Region A just went dark. What do your users see?

The setup

You run an active-passive service. Region A is the primary: stateless API and worker tiers spread across zones, and a relational database with a synchronous standby in a second zone. Region B is the standby: a smaller copy of the stateless tiers and a read replica fed by asynchronous replication from Region A. At peak traffic, Region A becomes unreachable. The interviewer asks you to walk through the next fifteen minutes.

This prompt is popular because it touches every layer, and because the honest answer contains bad news. Take it in order: detection, traffic, capacity, data, promotion and fencing.

1. Detection: who decides this is a regional outage?

Health checks from inside Region A may be failing, or they may be the thing that broke. A strong answer separates the signal from the decision. Signals come from several independent vantage points: synthetic probes from other regions and from outside the provider, error rates at the edge, and the provider's own health reporting. The decision to fail over a region is usually a human one, taken by an incident commander against a runbook, because a false-positive regional failover is an incident of its own: you accept data loss and a disruptive cutover to escape a problem that may have lasted ninety seconds.

Automating the traffic shift for the stateless tier is reasonable. Automating the database promotion is a much bigger decision, and you should say what evidence would justify it: for example, the primary unreachable from multiple independent vantage points for longer than a threshold, with the replica's lag known.

2. Traffic: how quickly do users actually move?

How traffic moves depends on how it arrives.

• A global load balancer behind an anycast address keeps the same IP for users and shifts traffic between backends on the provider's side. Once Region B is marked as the healthy backend, new connections move quickly, with no DNS change.
• DNS-based failover changes the address users resolve. It is only as fast as the record's TTL allows, and slower in practice: some resolvers, runtimes and clients cache longer than the TTL, and long-lived connections from mobile apps or service clients stay pinned to the old address until they break.

Then there is everything in front of you. CDN-cached pages keep serving, which is good for read traffic and confusing when the API behind them is in a different state. Mention it and move on.

3. Capacity: was Region B ready to take it?

Region B is described as a standby, so ask how warm it is. A warm standby runs a scaled-down copy of the stateless tiers and needs to scale out to peak, which takes minutes, during which the region is overloaded. A pilot light runs almost nothing but data replication, so the first minutes after failover are spent creating instances. In both cases three practical problems show up:

• Quotas. Account and project limits are set per region. A standby region sized for a quiet day may not be allowed to scale to peak. Raising quotas during an incident is slow; raise them in advance and test them.
• Correlated demand. If Region A belongs to a provider region that had a broad failure, other customers are failing over into the same nearby regions and competing for the same capacity.
• Cold caches. Region B's caches are empty, so the first wave of traffic falls through to a database that was until a moment ago a read replica. Load-shed or rate-limit until hit rates recover.

4. Data: what was lost, and what was stranded?

Replication to Region B is asynchronous, so at the moment Region A became unreachable, the replica was behind by some amount: the replication lag. Every write committed in Region A during that window is not in Region B. Those writes are either lost, if Region A's storage never comes back, or stranded, sitting in a database you can't currently reach. That window is your effective RPO, whether or not anyone wrote it down.

Two follow-ups to prepare for. First, lag is not constant: it grows under exactly the peak load when outages hurt most, so the RPO you measured on a quiet afternoon is optimistic. Monitor lag continuously and alert on it. Second, if the business needs an RPO of zero for some data, the answer is synchronous replication for that data only, accepting cross-region write latency on those operations, not for everything.

5. Promotion and fencing: make sure there is only one primary

Promoting the replica in Region B makes it writable. The danger is that Region A was only partly unreachable, or comes back, and its primary continues to accept writes. Now two databases each believe they are the primary and their histories diverge. This is split brain, and it is far harder to repair than an outage.

The fix is fencing: guaranteeing that the old primary cannot accept writes once the new one is promoted. Common mechanisms:

• Leases and epochs. The primary holds a time-limited lease from a coordination service and every write carries a monotonically increasing epoch or fencing token. Storage rejects writes with an older token, so a stale primary is refused even if it doesn't know it has been replaced.
• Forced isolation, the cloud equivalent of "shoot the other node in the head" (STONITH): stop the old primary's instances, revoke its credentials, or block its network path to clients and storage before promoting the new one.
• Routing. Ensure no application path can still reach Region A's database endpoint, so even a running old primary receives no writes.

When Region A recovers, the old primary must never rejoin as a primary. It is rebuilt as a replica of the new primary in Region B, after its unreplicated writes are dealt with.

6. The follow-up: Region A comes back with writes nobody replicated

"Region A returns forty minutes later. Its old primary holds writes that never reached Region B. What do you do with them?" This is the part of the case that separates candidates, because the right answer is partly technical and partly not.

1. Do not auto-merge. Region B has been taking writes for forty minutes. Some stranded writes will conflict with newer ones: the same order updated on both sides, an account balance changed twice, a username claimed by two people.
2. Quarantine and export. Keep the old primary fenced. Extract the writes after the last position Region B received (from the transaction log, or by diffing against the replication position) into a separate store for review.
3. Reconcile with rules. Writes that carry idempotency keys and don't conflict can often be replayed through the application, not directly into tables, so business validation still runs. Conflicting writes need explicit rules per entity type, and the rest go to humans.
4. Let the business decide. Whether a stranded order is honoured, refunded or cancelled is a product and legal decision, not a database one. The engineering job is to make the set of affected records small, exact and reviewable.
5. Tell customers. Some users were told an action succeeded when its effect is now uncertain. Plan the communication, then rebuild Region A as a replica and fail back on a schedule, not in a hurry.

Choosing a disaster recovery strategy

The case above is one point on a spectrum. Interviewers often ask you to justify where your design sits on it. The figures below are deliberately qualitative: real numbers depend on data size, automation and how often the plan has been tested.

Approximate and illustrative. Active-active is not just "two copies": it needs a data model that tolerates concurrent writes in two places, which is a design decision long before it is an infrastructure one.

Whatever you choose, the strongest single sentence you can add is about testing: an untested failover plan has an unknown RTO. Regular game days, restoring backups for real, and failing over a non-critical service on a schedule are what turn a strategy into a capability.

Common mistakes in cloud design interviews

• Starting with services. Opening with a list of product names before the requirements are clear signals memorisation. Describe the capability, then name a service if it helps.
• Treating multi-region as free resilience. A second region adds replication lag, failover risk, data transfer cost and operational load. Justify it with an RTO, RPO, residency or latency requirement.
• "It's managed, so it's highly available." Managed services fail too, and they fail in specific ways: zonal versus regional scope, failover times, quotas. Know the failure behaviour of anything you put on the critical path.
• Ignoring zone-loss headroom. Sizing for peak without capacity to absorb a lost zone means the system falls over the first time a zone does.
• Retries without limits. Unbounded retries turn a slow dependency into an outage. Pair retries with backoff, jitter, timeouts and circuit breakers.
• Forgetting data transfer. Cross-zone, cross-region and internet egress charges can dominate a design that moves a lot of data. Mention them when you place components.
• Security as an afterthought. Long-lived keys, public database endpoints and flat networks are easy to point out. Say how workloads get credentials before the interviewer asks.
• Never committing. Listing three options and choosing none is the most common way to run out of time. Pick one, state the tradeoff, and say what would make you change your mind.

Glossary

Frequently asked questions