What happens when one order is placed?

Why did one order arrive twice?

A small lunch-delivery kitchen in Pune takes twenty orders a day over WhatsApp. One weekend somebody builds it a website: one page, one button, Order. It runs on one rented server in the region the cloud console suggested, which happens to be on the US east coast.

At 6:10 the next morning a customer taps Order. Nothing happens. She taps again. Then “Confirmed” appears, twice, and the kitchen screen shows two identical orders forty seconds apart.

The server was fine. It answered every request it received, and each one took under 20 milliseconds of work. Time one order from the phone instead and it takes a little over a second. Nothing in that second was spent on the order. It was spent travelling: the phone is in Pune, the server is 13,000 km away, and a first visit is not one message but a short conversation, every turn of which crosses that distance.

Count the turns before reading on: resolve the name, open a connection, agree on encryption, then send the order. Each is a round trip. At roughly 250 ms each on a phone, four of them is a second, and a customer who sees nothing for a second taps again.

Before this kitchen can grow, you need to see every step one order takes.

Pause and predict One order took a little over a second, and the server was idle. Where did the time go?

One request is many trips

A request looks like one arrow: the phone asks, the server answers. A first request from a new visitor is a short conversation, and every turn of that conversation is a round trip.

Round-trip time (RTT)

How long it takes a message to reach the other side and a reply to come back, with no work done in between.

For example: Pune to a server in Mumbai: a few milliseconds on a good line. Pune to a server on the US east coast: roughly 200 to 300 ms on a phone.

Pause and predict A phone has never visited the site. The server uses TLS 1.2. How many round trips to the server happen before “Confirmed” comes back?

Step through the first order. Each arrow that crosses between the phone and the server costs one trip across that distance.

Four steps happen before the order exists at all:

• DNS turns the name dabba.example into an IP address. Usually one trip to a nearby resolver, and cached afterwards.
• The TCP handshake opens a connection. One round trip.
• The TLS handshake agrees on encryption keys. Two round trips with TLS 1.2, one with TLS 1.3.
• The HTTP request and response carry the order and the confirmation. One more round trip, plus whatever the server does.

With an RTT of 260 ms that is already about a second of pure travel. The server can be infinitely fast and the customer still waits.

A mental model

Two ideas follow from this picture, and both come back in every design later in the course.

Twenty database queries, one after another, are also trips: short ones, inside one machine, but they add up because each waits for the last.

Counting the trips

Here is the first order with the original settings: RTT 260 ms to the far region, 40 ms to the DNS resolver, TLS 1.2, 15 ms of server work, and 20 queries of 8 ms each.

Four round trips to the far region cost 1,040 ms: 83% of the total. The server and the database together cost 175 ms.

Now move the server to a region near Pune (RTT about 30 ms on a phone) and switch on TLS 1.3. A first visit costs 40 + 30 + 30 + 30 + 15 + 160 = 305 ms. A second order on the same open connection skips DNS, TCP and TLS: 30 + 15 + 160 = 205 ms.

Now the database is the biggest part. Replace the 20 queries with 2 (one to read the menu and prices, one to insert the order) and the repeat order costs 30 + 15 + 16 = 61 ms. That is twenty times faster than the first version, and the server hardware never changed.

Where should the one server live?

Moving the server is a weekend of work and a different price list. Leaving it where it is costs every customer a second on every first order. Both are real costs; here they are side by side.

Staying costs nothing today, but every customer pays for it with a slow first order, and the duplicate order shows that the slowness is already making customers tap twice and the kitchen cook two lunches.

Moving the server makes a second tap less likely, but it does not stop one. The full fix for the duplicate order is to let the server recognise a repeated tap of the same order and ignore it. That is done with an idempotency key, which Part 4 covers.

What to remember

1. A request is a conversation: DNS, the TCP handshake, the TLS handshake, then the HTTP request. Each crossing costs one round trip.
2. Latency is trips plus work. Distance sets a floor on each trip that no hardware can beat: about 1 ms of round trip per 100 km of fibre.
3. Reusing connections (keep-alive) and TLS 1.3 remove handshakes. Moving closer makes every remaining trip cheaper.
4. Sequential work adds up. Twenty quick queries in a row become the bottleneck once the network is fast.

Before you speed up the server, count the round trips.

In the real world

The simulator counts round trips the way the protocols define them. Real browsers soften the cost in several ways: they cache DNS answers, keep pools of open connections, resume earlier TLS sessions, and send many requests over one connection with HTTP/2 or HTTP/3. Large sites also end TLS at an edge server close to the user, a CDN (lesson 1.3).