Capacity planning and cost
How many machines will next year need, and what will they cost?
What does one order cost to serve?
On the last evening of the month two invoices sit on the kitchen table. The cloud provider: ₹108,178. The SMS gateway: ₹182,500. Beside them, the accountant’s own number: 1.2 million orders this month, about 40,000 a day.
Two plans for next year are already being argued over, and both are about machines. One says the orders will double, so move everything to Kubernetes with autoscaling and spot machines (spare cloud machines sold cheaply, which the provider can take back at short notice) and halve the bill. The other remembers last Diwali, when the site crawled at half past twelve, and wants enough machines for Diwali running all year, so nobody waits.
The accountant asks a different question first. What does one order cost to serve? And what will it cost next year, at twice the orders?
Nobody knows. The engineer who runs the system knows the CPU graph by heart and cannot answer. The cloud bill arrives as one number for the whole company, the SMS bill comes from a different vendor to a different email address, and nobody has ever divided either by orders.
When the kitchen made twenty tiffins a day, the cost of one box was known to the rupee: the dal, the rice, the steel box, the bus fare. If the cost went up, the owner knew which item had gone up. Nobody can say the same about what the servers and text messages cost for one box.
Nor can anyone say how many machines next year needs, because the only plan so far is to add a server when the CPU graph turns red. Do the division before reading on: two invoices, 1.2 million orders. Which bill is most of the answer?
Pause and predict Cloud ₹108,178 and SMS ₹182,500, for 1.2 million orders. What does serving one order cost, and which bill is most of it?
Adding a server when the graph turns red
The current plan is reactive: watch CPU, and when it passes 80% at lunch, add an app server. It has worked so far.
Pause and predict Orders double next year. Which part of the system cannot be rescued in time by adding a server when CPU passes 80%?
A mental model
- Headroom
- The spare capacity you deliberately keep at the peak: the difference between how busy your machines are and how busy they could be before latency breaks.
- For example: One instance serves 200 requests a second within the p99 target. Planning it at 60% means counting on 120 at the busiest minute of lunch, and keeping 80 for surprises and for a lost zone.
- Unit cost
- What it costs to serve one unit of the business, here one order: every line of the bill that grows with orders, divided by the number of orders.
- For example: ₹290,678 for 1.2 million orders is about ₹0.24 an order. If next month is ₹0.27, something got more expensive per order, whatever the total did.
The plan, by hand
| Quantity | Working | Result |
|---|---|---|
| Average requests, today | 40,000 x 60 / 86,400 | 27.8/s |
| Lunch peak | 27.8 x 6 | 166.7/s |
| Usable per instance | 200 x 0.6 | 120/s |
| Instances, surviving one of three zones | 166.7 / 120 / (2/3) = 2.08 | 3 |
| Orders a day in a year, at 8% a month | 40,000 x 1.0812 | 100,727 |
| Instances then | 419.7 / 120 / (2/3) = 5.25 | 6 |
| Peak database writes, today | 40,000 / 86,400 x 6 x 12 | 33.3/s |
| Month the current size (60/s) runs out | 33.3 x 1.087 = 57.1; x 1.088 = 61.7 | month 8 |
| Cost per order, today | ₹290,678 / 1,216,667 orders | ₹0.239 |
| Without the SMS line | ₹108,178 / 1,216,667 | ₹0.089 |
Each row is a back-of-envelope estimate: complexity analysis with the real constants filled in (complexity in Coding Patterns).
The fleet doubles in a year and the bill grows with it, yet the cost per order falls slightly, from ₹0.239 to ₹0.223 in month 12, because storage and the database grow in steps, not per order. A cost per order that rises while orders rise is the warning sign: something is growing faster than the business.
Buy for Diwali, or buy for the floor?
Both plans want the same thing, no slow lunch, and disagree about how to pay for it. Here are both paths a year from now, in month 12.
The second path cuts compute by 61% in month 12. Over the whole year it saves about ₹220,000 out of ₹5.6 million, under 4%, because compute is a small line. That is the most useful number in the fork: neither path touches the SMS line, which is 67% of month 12’s bill. Halving SMS (sending push notifications to customers who have the app, lesson 4.5) saves ₹91,250 a month today, more than the entire compute line.
The fixed fleet is right about one thing, and the decision keeps it: the lunch peak is known in advance, so the extra instances are started on a schedule, not left to an autoscaler that reacts after the queue has grown.
What to remember
- Capacity planning is a forecast (organic growth and dated, inorganic events) turned into machines by a load-tested number per instance, a target utilisation (headroom) and N + 1 across zones.
- Plan further ahead than the slowest thing you must buy. App servers arrive in minutes; a database resize takes weeks.
- Divide every line of the bill by orders. Cost per order tells you what the total cannot: whether the business is getting cheaper or dearer to run.
- Buy the always-on floor with a commitment, add the peak on a schedule, and put only work that can restart on spot capacity.
- Attack the biggest line first. It is often not the machines.
Divide the bill by orders before you argue about machines.
In the real world
The simulator is a spreadsheet with sliders. Real plans add what it leaves out: several services with different load-tested numbers, caches that change the database load (lesson 3.7), data that grows faster than orders, reserved database capacity, network charges between zones, support plans, and the people who run it, often the largest cost on the list.
The measurements come from the systems you have already built: the request rates and p99s from monitoring (lessons 5.7 and 5.8), the bill tagged by service, and a load test before every festival. A plan is a set of numbers you re-check every month, never a document written once.