Cloud interview questions and answers

What is the difference between an availability zone and a region, and when do you need more than one region?

What it tests: Whether you understand failure domains and can tie a multi-region decision to a requirement rather than a habit.

• A region is a geographic area with its own control plane and services. An availability zone is one or more isolated data centres inside a region, with separate power, cooling and networking, joined to the other zones by low-latency links.
• Spreading a service across zones protects against the loss of a data centre and is relatively cheap, because round trips between zones are short enough for synchronous replication.
• Going multi-region protects against a regional outage, a regional control plane failure or a bad regional deployment, and it can bring the service closer to distant users. It costs real money and adds real complexity.
• The deciding inputs are the availability target, the recovery time and recovery point objectives, data residency rules and latency for distant users. If none of them demands it, a well-built multi-zone design is usually the right answer.
• Replication between regions is normally asynchronous, so a multi-region design forces an explicit decision about how much data you can lose and where writes go.

Common trap: Saying “multi-region for high availability” without naming the failure it covers or what it does to write consistency.

Your service must survive a sudden 10x traffic spike. How do you design for it?

What it tests: Capacity reasoning, the limits of elasticity, and protecting the system when scaling is not fast enough.

• Start by finding what does not scale elastically: databases, connection pools, third-party APIs, account quotas and anything with a warm-up time.
• Keep the request path stateless so compute can scale horizontally, and put caches and a CDN in front of read-heavy paths to cut load on the origin.
• Autoscaling reacts in minutes, not seconds. For a known event, pre-scale or schedule capacity. For an unknown one, keep headroom and use instances or containers that start quickly.
• Move work that does not need to be synchronous onto a queue, so the spike becomes a backlog instead of an outage.
• Protect the core with rate limits, timeouts and load shedding, and decide in advance which features degrade first.
• Check account and regional quotas before the event. Hitting a quota during the spike looks exactly like a broken autoscaler.

Common trap: Answering “add autoscaling” and stopping, as if the database and every downstream dependency scale with the web tier.

When should two services communicate through a queue instead of a synchronous API call?

What it tests: Coupling, failure isolation, and the delivery semantics you inherit from a message broker.

• Use a synchronous call when the caller needs the answer to continue and a user is waiting for it.
• Use a queue when the work can finish later, when the consumer is slower or less reliable than the producer, or when one event fans out to several consumers.
• A queue absorbs bursts and lets producer and consumer fail and scale independently. In exchange it adds end-to-end latency and one more component to operate and monitor.
• Most managed queues deliver at least once, so consumers must be idempotent and the design needs a dead-letter queue for messages that keep failing.
• Watch the age of the oldest message, not only the queue depth. A growing age is the clearest sign that consumers are falling behind.

Common trap: Claiming the queue gives exactly-once processing without explaining deduplication or idempotency in the consumer.

How would you isolate tenants in a multi-tenant SaaS platform?

What it tests: Isolation models, noisy neighbours, and how the choice affects cost, security and operations.

• Isolation is a spectrum: shared everything with a tenant ID on every row, pooled compute with a schema or database per tenant, and a fully separate stack or account per tenant.
• Shared models are the cheapest and simplest to run, but every query needs tenant-aware authorisation and the platform needs strong protection against noisy neighbours.
• Siloed models give the clearest security and compliance boundary and an exact cost per tenant, but they multiply the number of things to deploy, patch and monitor.
• Many platforms use tiers: pooled by default, with dedicated resources for large or regulated customers.
• Enforce isolation in more than one layer, for example tenant-scoped credentials plus row-level security in the database, and apply per-tenant quotas and rate limits.

Common trap: Relying on application code to remember the tenant filter on every query, with nothing underneath to catch a mistake.

Explain strong and eventual consistency with a cloud example. How do you choose between them?

What it tests: Whether you can link a consistency model to a business consequence and to latency.

• With strong consistency, a read after a successful write always returns that write. With eventual consistency, replicas converge, but a read can briefly return older data.
• Strong consistency across distance costs latency, because replicas in more than one place must agree before a write succeeds. During a network partition, a strongly consistent system may refuse requests rather than return a wrong answer.
• A payment ledger or an inventory count needs strong consistency. A product catalogue, a like counter or a social feed can tolerate a few seconds of staleness.
• Many cloud databases let you choose per request, for example a strongly consistent read that costs more or goes to the leader, against a cheaper read from any replica.
• Choose by asking what a stale read costs the business. If it causes double spending or overselling, pay for strong consistency on that path only.

Common trap: Treating the CAP theorem as one choice for the whole system, rather than a choice per operation and per failure mode.

When would you choose virtual machines, containers or serverless functions for a workload?

What it tests: Matching the compute model to the workload's runtime, scaling pattern and operational cost.

• Virtual machines suit long-running, stateful or specialised workloads, licensed software, and anything that needs control of the kernel or hardware. You manage the operating system.
• Containers package an application with its dependencies, start quickly and pack densely. On Kubernetes or a serverless container platform they suit most long-running services.
• Functions suit short, event-driven tasks with spiky or low traffic. You pay per invocation and can scale to zero, but you accept execution time limits, cold starts and less control of the runtime.
• Compare total cost at the expected utilisation. At steady high load, provisioned capacity is usually cheaper than paying per request.
• Operational cost matters as much as the bill. A small team can be better served by a managed platform even at a higher unit price.

What do CPU and memory requests and limits do in Kubernetes, and what goes wrong when they are set badly?

What it tests: Scheduling versus enforcement, and the difference between compressible and incompressible resources.

• Requests are what the scheduler uses to place a pod: a node is chosen only if the sum of requests fits its allocatable capacity. Requests also determine the pod's quality-of-service class.
• Limits are enforced at runtime. CPU is compressible, so a container that reaches its CPU limit is throttled. Memory is not, so a container that exceeds its memory limit is killed.
• Requests set too low lead to overcommitted nodes, contention and evictions under memory pressure. Requests set too high waste capacity and make the cluster autoscaler add nodes that sit mostly idle.
• Tight CPU limits can cause latency spikes from throttling even when the node has spare CPU. This is why many teams set CPU requests but leave CPU limits off for latency-sensitive services.
• Base requests on measured usage plus headroom, and set the memory request close to the memory limit for workloads that must not be evicted.

Common trap: Assuming a container that averages well below its CPU limit cannot be throttled. Throttling is enforced over short periods, so bursty workloads hit it.

A deployment's new pods are stuck in Pending. How do you debug it?

What it tests: A systematic path through the scheduler rather than guesswork.

• Pending means the pod has not been placed on a node, so start with kubectl describe pod and read the scheduler's events.
• Common causes: no node has enough unreserved CPU or memory for the requests, node selectors or affinity rules match no node, or the pod does not tolerate a node's taints.
• Storage can block scheduling too: a PersistentVolumeClaim that cannot bind, or a volume that exists in a different zone from the available nodes.
• If the cluster autoscaler should have added a node, check its status and logs. It may be at its maximum size, blocked by a cloud quota, or unable to find a node group that satisfies the constraints.
• If the pods do not exist at all, the problem is earlier: an admission webhook or a namespace ResourceQuota rejected them, and the error appears in the ReplicaSet's events.

Common trap: Confusing Pending with CrashLoopBackOff. A crashing container has already been scheduled and started. A Pending pod never reached a node.

Explain how horizontal pod autoscaling and cluster autoscaling work together.

What it tests: The two control loops, their combined delay, and how they interact.

• The Horizontal Pod Autoscaler changes a workload's replica count based on metrics such as CPU utilisation relative to requests, or custom and external metrics like queue length.
• When new replicas cannot be scheduled they go Pending, and the cluster autoscaler or a node provisioner such as Karpenter adds nodes that fit them. It also removes underused nodes when their pods can move elsewhere.
• The delays stack up: metric collection, the autoscaler's decision interval, node provisioning, image pulls and application warm-up. Together they can take several minutes.
• Because utilisation is measured against requests, wrong requests produce wrong scaling. Scaling on a signal closer to demand, such as requests per second or queue length, often works better than CPU.
• If minutes of delay are unacceptable, keep headroom, for example with low-priority placeholder pods that are preempted as soon as real pods need the space.

Common trap: Expecting the autoscalers to absorb a spike that arrives faster than new nodes can be provisioned.

What causes cold starts on serverless platforms, and how do you reduce their impact?

What it tests: Understanding of the execution lifecycle and the trade-offs of each mitigation.

• A cold start happens when the platform must create a new execution environment: allocate capacity, load the code or image, start the runtime and run your initialisation code.
• They occur on the first request, after idle periods, and whenever concurrency rises beyond the warm environments available.
• Make each cold start cheaper with smaller packages, fewer dependencies, lazy initialisation and faster-starting runtimes.
• Make them rarer for users by keeping a minimum number of warm instances, which trades money for latency, or by moving the most latency-critical paths off functions.
• Measure cold-start frequency and the latency tail, such as p99, not the average, because cold starts live in the tail.

Walk me through a request travelling from a user's browser to a database in a private subnet.

What it tests: End-to-end understanding of DNS, TLS, load balancing, routing and security controls.

• DNS resolves the hostname, often to a CDN or a global load balancer address. The browser opens a TCP or QUIC connection and negotiates TLS.
• The edge or load balancer terminates TLS, applies WAF rules and forwards the request to a healthy backend, re-encrypting it if the design requires encryption in transit inside the network.
• The application runs in a private subnet with no public IP address. Firewall rules or security groups accept traffic only from the load balancer.
• The application reaches the database over the private network, allowed by a rule that names the application's security group or identity rather than a broad address range.
• Outbound internet calls from the private subnet go through a NAT gateway or an egress proxy, which is a separate path with its own controls and its own cost.

Common trap: Not knowing where TLS terminates, and therefore where traffic is unencrypted and who can see it.

How do you plan IP address ranges for VPCs across many accounts and regions?

What it tests: Foresight about growth, peering, hybrid connectivity and Kubernetes address consumption.

• Allocate from one central plan, ideally an IP address management tool, so ranges never overlap. Overlapping networks cannot be peered or routed to each other directly later.
• Reserve space for on-premises networks, partner connections and future regions before anyone needs them.
• Size subnets generously for workloads that consume many addresses. Kubernetes networking that gives every pod a VPC address can exhaust a subnet long before the nodes run out of CPU.
• Use a consistent scheme, such as one block per region and one per environment, so routes can be summarised and firewall rules stay readable.
• The private ranges defined in RFC 1918 are finite. Plan for IPv6, or for secondary ranges used only by pods, before the space runs out.

Common trap: Giving every team the same default range and discovering the conflict when the first peering request arrives.

What is the difference between a layer 4 and a layer 7 load balancer, and when do you use each?

What it tests: Protocol awareness and the operational consequences of each choice.

• A layer 4 load balancer routes TCP or UDP connections by address and port. It does not read the request, so it is fast, works with any protocol and can pass encrypted traffic straight through to the backend.
• A layer 7 load balancer understands HTTP. It can route by host, path or header, terminate TLS, apply WAF rules, retry failed requests and balance individual requests rather than connections.
• Use layer 4 for non-HTTP protocols, very high throughput, or when TLS must terminate on the backend. Use layer 7 for web APIs, path-based routing and microservices.
• Long-lived connections such as gRPC streams or WebSockets behind a layer 4 balancer can leave load uneven, because balancing happens once per connection.
• Many designs use both: a network load balancer at the edge and a layer 7 proxy or service mesh inside.

Why can DNS-based failover take longer than the record's TTL suggests?

What it tests: How DNS caching really behaves and why failover design must allow for it.

• DNS answers are cached by resolvers and clients for the record's TTL, so changing a record does not move traffic until those caches expire.
• Some resolvers, operating systems and application runtimes keep answers longer than the TTL, and clients with open connections keep using the old address until the connection closes.
• Health-checked DNS also needs time to detect the failure, roughly the probe interval multiplied by the failure threshold, before the record changes at all.
• Lower TTLs speed up failover but increase query volume and dependence on the DNS provider. A common compromise is a short TTL only on the records used for failover.
• A global load balancer with an anycast address avoids the DNS change entirely, which is why it usually fails over faster.

Common trap: Setting a 60-second TTL and promising 60-second failover.

Your bill shows large NAT gateway charges. What is happening, and how do you fix it?

What it tests: Knowledge of egress paths and cost-aware network design.

• Managed NAT gateways typically charge per hour and per gigabyte processed, so heavy outbound traffic from private subnets becomes expensive.
• Often the traffic is not going to the internet at all. It is going to the provider's own services, such as object storage or a container registry, through the NAT.
• Fix that with private endpoints to those services, so the traffic stays on the provider's network and skips the NAT.
• Check for cross-zone paths too. Workloads in one zone using a NAT gateway in another pay for the cross-zone transfer as well and lose zone independence.
• Use flow logs to find the biggest senders, then cache dependencies such as container images and packages closer to the workloads.

How do you implement least privilege in cloud IAM without slowing the team down?

What it tests: Practical identity design, not just the definition.

• Grant permissions to roles rather than individual users, and have people and workloads assume those roles with short-lived credentials.
• Start from narrow, task-based roles, then use access analysis tools that compare granted permissions with those actually used, and remove the rest.
• Set guardrails at the organisation level, such as policies that deny disabling audit logs or making storage public, so any single mistake is bounded.
• Give developers broad access in sandbox accounts and tight access in production, with a just-in-time, audited path to elevated access when it is needed.
• Manage policies as code with review, so every change is visible and reversible.

Common trap: Removing access people genuinely need, which pushes them to share admin credentials as a workaround.

Why should workloads use workload identity instead of long-lived access keys?

What it tests: Credential lifecycle and how cloud identity federation works.

• Long-lived keys leak through code, logs, images and laptops, and they stay valid until someone notices and rotates them.
• Workload identity lets the platform issue short-lived credentials to a VM, a function or a Kubernetes service account, so there is nothing to store or rotate by hand.
• In Kubernetes, a pod's service account identity is exchanged for cloud credentials scoped to one role, so each workload holds only its own permissions.
• Federation extends the same idea outside the cloud. A CI pipeline can exchange its signed identity token for temporary cloud credentials instead of storing keys as secrets.
• Audit logs then show which workload acted, instead of one shared key used by many.

Explain envelope encryption and why cloud key management services use it.

What it tests: Practical cryptographic design and the trade-offs of key ownership.

• Data is encrypted with a data encryption key, and that key is itself encrypted with a key encryption key held in a key management service or hardware security module.
• The encrypted data key is stored next to the data. To read it, the service asks the KMS to decrypt the data key and then decrypts the data locally.
• Bulk encryption stays fast and local, while the root key never leaves the KMS and every use of it is authorised and logged.
• Rotating the key encryption key does not mean re-encrypting all the data. Older key versions stay available for decryption, and new data keys are wrapped with the new version.
• Customer-managed keys add control, such as cutting off access by disabling the key, but you then own that key's availability. A destroyed key means data you can never read again.

Common trap: Answering “it is encrypted at rest” without saying who controls the key and who is allowed to use it.

How should applications on the cloud handle secrets such as database passwords and API keys?

What it tests: Secure handling of credentials across the build and runtime lifecycle.

• First remove secrets where you can, using workload identity and IAM-based database authentication instead of passwords.
• Store the remaining secrets in a managed secrets manager, encrypted with KMS, with access granted to individual workload identities.
• Keep secrets out of source code, container images and configuration repositories. Inject them at runtime or mount them as files.
• Rotate secrets automatically, and build applications that pick up a new value without a restart.
• Audit access to secrets and alert on unusual reads.

A storage bucket holding customer data was made public. What do you do in the first hour, and how do you prevent it happening again?

What it tests: Incident priorities and preventive controls at organisation scale.

• Contain first: block public access on the bucket, and preserve evidence rather than deleting anything.
• Establish the exposure window and what was read, using access logs and audit trails. If access logging was off, say so, because it changes what you can prove.
• Bring in security, legal and privacy early. Notification obligations can depend on what data was exposed and for how long.
• Find the root cause: who changed the policy, through which path, and why no control stopped it.
• Prevent a repeat with organisation-wide public access blocks, policy-as-code checks in the deployment pipeline, and configuration monitoring that alerts within minutes.

Common trap: Quietly fixing the bucket and moving on without establishing what was accessed.

When do you use object, block and file storage?

What it tests: Matching the storage type to access pattern, sharing and durability.

• Block storage is a virtual disk attached to an instance. It has low latency and suits databases and boot volumes.
• File storage is a shared file system with familiar file semantics that many instances can mount at once. It suits legacy applications and shared content.
• Object storage holds objects accessed over an HTTP API. It scales almost without limit, is durable and cheap, and suits backups, media, data lakes and static assets.
• Objects are replaced rather than edited in place, so object storage is a poor fit for workloads that update small parts of a file frequently.
• Durability differs too. Most object storage classes replicate across zones by default, while a block volume usually lives in a single zone.

How do you choose between a relational database, a key-value store and a data warehouse?

What it tests: Data modelling judgment driven by the workload, not by fashion.

• Start from the access patterns: which queries run, how often, at what latency, and with what consistency.
• Relational databases suit transactional data with relationships, constraints and ad hoc queries. They scale up easily and scale out with more effort.
• Key-value and wide-column stores suit very high request rates by known keys with predictable latency, but you must design the keys around the queries in advance.
• Data warehouses suit analytical queries that scan large volumes, using columnar storage and separate compute. They are not built for high rates of single-row transactions.
• Many systems use more than one, with change data capture streaming changes from the transactional store into analytics.

Common trap: Choosing a NoSQL store for scale before checking whether one well-indexed relational database can handle the load.

One partition of your distributed database is overloaded while the others are idle. What do you do?

What it tests: Understanding of partitioning and key design under skew.

• A hot partition appears when traffic concentrates on one key or range: a popular item, a timestamp-ordered key, or one tenant much larger than the rest.
• Confirm it with per-key or per-partition metrics before changing anything.
• Redesign the key to spread load, for example by adding a suffix that splits one logical key across several physical keys, and combine the parts on read.
• Put a cache in front of read-heavy hot keys, and buffer or batch writes to write-heavy ones.
• Avoid keys that only ever increase, such as raw timestamps, in range-partitioned stores, because every new write lands on the same partition.

How do you design backups, and how do you prove you can restore them?

What it tests: Whether you treat backup as a recovery capability with a measured outcome.

• Define the requirement first: the recovery point objective, how much data you can lose, and the recovery time objective, how long a restore may take.
• Combine snapshots or full backups with continuous logs where the database supports point-in-time recovery.
• Protect backups from the failures they exist for. Copy them to another region and to a separate account with restricted access, and use immutable storage against ransomware and accidental deletion.
• Test restores regularly and automatically, measure how long they take, and check the restored data, not only that the job succeeded.
• Write the runbook down, because a restore under pressure is the worst time to work it out.

Common trap: Treating a successful backup job as proof that the data can be recovered.

A customer requires that their data stays in the EU. What does that mean for your architecture?

What it tests: Turning a regulatory requirement into concrete architecture, including the hidden data paths.

• Clarify the requirement: storage location only, or also processing, support access, backups, logs and subprocessors.
• Deploy the data stores, their replicas and their backups in EU regions, and pin the tenant to those regions in routing.
• Check the less obvious paths: logs, traces, analytics pipelines, search indexes, caches, and AI model endpoints that may process data in another region.
• Control operational access, such as whether support engineers outside the EU can see the data, and use customer-managed keys where the contract calls for them.
• Make the guarantee enforceable and testable, with policies that deny creating resources outside allowed regions and an up-to-date map of data flows.

What are SLIs, SLOs and error budgets, and how do they change engineering decisions?

What it tests: Whether you can use reliability targets to make trade-offs, not just define the terms.

• An SLI is a measurement of user experience, such as the share of requests that succeed within 300 milliseconds.
• An SLO is the target for that measurement over a window, such as 99.9 percent over 30 days. An SLA is a contractual promise, usually looser, with penalties attached.
• The error budget is the unreliability you are allowed. At 99.9 percent over 30 days it is about 43 minutes of complete downtime, or the equivalent in failed requests.
• While budget remains, teams ship faster and take more risk. When it is spent, priority shifts to reliability work until it recovers.
• Alerting on burn rate, how fast the budget is being consumed, catches serious problems quickly without paging anyone for small blips.

p99 latency doubled after a deploy, but the average looks fine. How do you investigate?

What it tests: Methodical use of observability under pressure.

• The tail matters because a page that makes many requests is likely to hit at least one slow one. Averages hide it.
• Check the correlation with the deploy, comparing canary against baseline. If the impact is significant and the cause is not obvious, roll back first.
• Use traces to find which span grew in the slow requests: a downstream call, lock contention, garbage collection, a cache miss path or a new query.
• Break latency down by endpoint, zone, instance, tenant and payload size. One bad host or one large tenant often explains the tail.
• Check saturation signals such as CPU throttling, connection pool waits and queue depth, which usually show in the tail before they move the average.

Compare backup and restore, pilot light, warm standby and active-active disaster recovery.

What it tests: Matching recovery strategies to RTO, RPO and budget.

• Backup and restore is the cheapest, but recovery takes the longest, because infrastructure must be created and data restored before anything serves traffic.
• Pilot light keeps data replicated and a minimal core running in the recovery region, then scales the rest up during a disaster. Recovery speed depends on how well that scale-up is automated.
• Warm standby runs a scaled-down copy of the full stack that can take traffic quickly and then grow. It recovers faster and costs more.
• Active-active serves traffic from both regions all the time, so recovery is close to immediate. The hard part is handling writes in more than one region without conflicts.
• Choose from the business recovery objectives and rehearse the failover regularly. An untested plan has an unknown recovery time.

You are paged for a major outage. What do you do in the first 30 minutes?

What it tests: Structured response, communication and mitigation-first thinking.

• Acknowledge the page, assess user impact and declare an incident with a severity, so the right people join.
• Assign roles: an incident commander who coordinates, responders who investigate, and someone who updates stakeholders and customers.
• Prioritise mitigation over root cause. Roll back the last change, shift traffic, fail over or shed load to restore service.
• Keep a timeline of actions and observations as you go, which makes the postmortem accurate.
• Communicate on a regular cadence even when nothing has changed, and declare the incident resolved only when user-facing metrics have recovered.

Common trap: Spending the first twenty minutes looking for the root cause while users are still affected.

What is the difference between metrics, logs and traces, and how do you use them together?

What it tests: Using telemetry as a connected system for diagnosis.

• Metrics are numeric time series. They are cheap to store and query, and they suit dashboards and alerts on rates, errors, latency and saturation.
• Logs are detailed records of events. They show exactly what happened, but they are expensive at volume and hard to aggregate unless they are structured.
• Traces follow one request across services, showing where the time went and which dependency failed.
• A typical path: an alert fires on a metric, a filter or exemplar leads to slow traces, and the trace ID leads to the logs from exactly those services.
• Use consistent identifiers and standard instrumentation such as OpenTelemetry so the signals connect, and control cost with sampling and retention.

How would you work out the cloud cost of a single request or a single customer?

What it tests: Connecting architecture to business metrics.

• Start with tags or labels so every resource maps to a service, team and environment, and split shared costs by an agreed rule.
• Divide each service's cost by a business unit of work: requests, active users, orders or gigabytes processed.
• Include the hidden parts: data transfer, logging and monitoring, managed service fees, idle capacity and non-production environments.
• Track the unit cost over time. Total spend can rise healthily with growth while cost per request falls. The opposite is the warning sign.
• Use the number in design decisions, for example to compare the saving from a cache or a cheaper storage class with the engineering time it takes.

When do you use on-demand capacity, committed-use discounts and spot capacity?

What it tests: Matching purchasing models to workload characteristics.

• On-demand capacity is flexible and the most expensive per hour. Use it for unpredictable or short-lived needs.
• Commitments, such as reserved instances, savings plans or committed-use discounts, lower the price in exchange for committing to a level of usage for one or three years. Cover the steady baseline with them.
• Spot or preemptible capacity is much cheaper but can be reclaimed at short notice. Use it for fault-tolerant work: batch jobs, CI runners, stateless workers and training jobs that checkpoint.
• Commit only to usage you are confident of, because unused commitment is pure waste.
• Spread spot capacity across instance types and zones to reduce interruptions, and handle the interruption notice gracefully.

Which data transfer costs surprise teams, and how do you design around them?

What it tests: Knowing where bytes are billed and designing with that in mind.

• Internet egress is the obvious charge, but traffic between zones and between regions inside the provider is often billed too.
• Chatty microservices spread across zones, replication between regions, and log pipelines that ship everything to another region add up quietly.
• NAT gateways and some load balancers charge for data processed, on top of the transfer itself.
• Design around these charges with zone-aware routing where availability allows, compression, a CDN for user-facing content, and private endpoints to cloud services.
• Measure before optimising. Flow logs and the provider's cost reports by usage type show where the bytes go.

Common trap: Tuning instance sizes while a cross-region replication stream costs more than the instances.

How do you rightsize a fleet without hurting performance?

What it tests: Safe, data-driven cost optimisation.

• Collect utilisation over a representative period, including peaks, for CPU, memory, network and disk, not CPU alone.
• Downsize gradually, starting with clearly over-provisioned and non-critical workloads, and watch latency and error objectives after each change.
• Consider newer instance generations or different processor architectures, which often give better price-performance once tested.
• Remove what is idle: non-production environments outside working hours, unattached volumes, old snapshots and unused load balancers.
• Make it continuous, with automated recommendations and a named owner, because usage drifts after every release.

Engineering teams don't feel responsible for cloud cost. How do you change that?

What it tests: Organisational influence and FinOps practice.

• Make cost visible per team through allocation tags and dashboards, ideally in tools engineers already use.
• Start with showback, reporting costs to teams, before chargeback, billing them internally, so teams can learn before there is a penalty.
• Give teams unit cost metrics and budgets with anomaly alerts, so a surprise reaches its owner within a day instead of at month end.
• Put cost into the design process, with an estimate in design reviews and in infrastructure pull requests.
• Recognise savings that keep reliability intact, and avoid policies that cut cost by quietly lowering availability.

How should an AI agent authenticate to the tools and systems it calls?

What it tests: Identity design for a non-human actor that takes instructions from untrusted text.

• The agent should not hold long-lived keys or broad service credentials. Each tool call should use short-lived, narrowly scoped credentials.
• When the agent acts for a user, use delegated, on-behalf-of authorisation, so it can never do more than that user is allowed to do.
• Route tool access through a gateway that enforces policy, scopes credentials per tool and per task, and logs every call.
• Require human approval for irreversible or high-impact actions such as payments, deletions or messages sent outside the company.
• Treat what the agent reads, such as web pages, emails and documents, as untrusted input that may contain instructions, and never let that content widen its permissions.

Common trap: Giving the agent a service account with broad access because it is “just an internal tool”.

An agent's tool call times out. How do you make sure a retry does not repeat the action?

What it tests: Distributed systems fundamentals applied to agent workflows.

• A timeout does not tell you whether the action happened, so a blind retry can repeat a side effect.
• Give every side-effecting call an idempotency key derived from the task and step, so the receiving system applies it at most once.
• Run multi-step tasks on a durable execution or workflow engine that records each step's result and resumes from the last completed step after a crash.
• For actions that cannot be made idempotent, check the current state before retrying, or define a compensating action that undoes the step.
• Cap retries with backoff, and hand tasks that stay stuck to a human instead of letting the agent loop.

How do you stop an agent from running up an unbounded model bill?

What it tests: Operational control of a workload whose cost depends on model behaviour.

• Set limits per task, such as maximum steps, maximum tokens and a wall-clock timeout, and enforce them in the runtime rather than in the prompt.
• Attribute cost by tenant, user and task through tracing, so a runaway pattern shows up quickly.
• Send each step to the smallest model that handles it well, reuse repeated context with prompt caching where the provider supports it, and drop context a step does not need.
• Detect loops, such as the same tool called repeatedly with the same arguments, and stop them.
• Alert on cost anomalies, and keep provider-side quotas or spending limits as a backstop.

Common trap: Asking the model in the prompt to be economical and treating that as a control.

Your agent can write and run code. How do you run that code safely on cloud infrastructure?

What it tests: Isolation and blast-radius thinking for untrusted code.

• Run the code in an isolated, ephemeral sandbox, such as a microVM or a container with a hardened kernel boundary, created per task and destroyed afterwards.
• Give the sandbox no ambient cloud credentials and no route to the instance metadata service. Pass in only the data the task needs.
• Deny network egress by default and allowlist specific destinations when a task needs them, to limit data exfiltration and abuse.
• Enforce CPU, memory, disk and time limits, so a runaway or malicious program cannot exhaust shared resources.
• Log what ran and what it produced, and check outputs before they reach other systems.

Common trap: Running agent-generated code in the same container as the agent, with the agent's credentials.

How do you know a new version of your agent is better before rolling it out?

What it tests: Evaluation discipline and release engineering for non-deterministic systems.

• Keep a fixed evaluation set of representative tasks with known good outcomes, including past failures and edge cases.
• Grade with deterministic checks where possible, such as whether the right tool was called with valid arguments, and with rubric or model-based grading where judgment is needed, calibrated against human review.
• Compare quality, cost per task, latency and safety violations between versions, not quality alone.
• Roll out gradually behind a canary, compare online metrics, and keep a fast rollback path.
• Capture production traces so that new failures become new evaluation cases.

Common trap: Judging a new prompt or model by reading a handful of outputs.