What happens when everyone turns up to your website at once
Tech Titbit Thursday, episode three: load balancing, scaling out, scaling up, health checks and autoscaling, explained with supermarket queues, a van and a cafe at lunchtime.

A website that works fine for ten people can fall over for ten thousand. Same code, same server, same everything. The only thing that changed is the crowd.
That's the problem behind this week's Tech Titbit Thursday, my weekly series where I pick five tech words and explain them in plain language, the way I'd explain them over a coffee. Quick recap of last week: caching is milk in the fridge. A CDN keeps stock everywhere, and edge caching is the corner dairy with the pies. Invalidation wipes the old prices off the board. And cold starts are the coffee machine warming up.
Last week was about not doing the same work twice. This week is about what happens when there's simply more work than one machine can handle. These are the words your AI coder reaches for when it talks about handling a rush, so it helps to know what they mean before you agree to anything.
One: load balancing
One address out front, lots of servers behind it, and something in the middle sharing out the requests.
Think of the person at the supermarket waving you to the shortest queue. You walked in through one door. You don't need to know how many checkouts there are, or which one is free. Someone does that bit for you.
For your website, it means visitors type one address and never see the machinery. Behind the scenes the work gets spread around, so no single server is the one drowning.
Two: horizontal scaling
If it's busy, add more servers. That's scaling out.
Picture opening more checkouts instead of asking one person to scan faster. Each checkout is ordinary. There are just more of them.
This only works properly with a load balancer in front, because someone still has to wave people towards the new tills. That's why the two words tend to turn up in the same sentence.
Three: vertical scaling
Same server, just bigger, with more memory and more processing power. That's scaling up.
It's swapping the hatchback for a van. Simple, and often the quickest fix, because nothing about how the site is set up has to change. But there's only so big a van gets. At some point you can't buy a bigger one, and you're back to needing more vehicles.
Four: health checks
The load balancer keeps asking each server, "are you okay?" If one stops answering, it stops getting traffic.
In supermarket terms, that's closing a till when the scanner breaks. Nobody wants to be sent to the checkout that can't scan anything, and a good supervisor notices before the queue does.
Of the five, this one gets the least attention and probably deserves the most. Extra servers don't help much if one of them is quietly broken and still being handed customers.
Five: autoscaling
Servers get added when traffic climbs and removed when it drops, all on their own.
Cafes do this every day. They call in extra staff for the lunch rush, then send them home once it goes quiet. You're not paying for a full crew in the dead hours of the afternoon, and you're not short-handed at half twelve either.
Out, not just up
My view: scaling up is the tempting first answer, because it's one change and nothing else has to move. Sometimes it's the right call. But a site that stays standing through a rush usually does it by scaling out, with health checks quietly taking the broken tills out of the line. One bigger van is still one van. If it gets a flat, everyone waits.
Mr. Mummu, who turns up in these posts now and then, learnt this at his nephew's engagement party. He was in charge of samosas. The guest list doubled overnight, and his answer was to buy the biggest pot the Indian grocer had, because one big pot "saves washing up, which is thirty minutes of my life". The pot didn't fit on any element in the house. He spent the afternoon balancing it across two burners while the samosa queue reached the letterbox. In the end his sister set up three ordinary pans on three ordinary burners and had the lot fried before the speeches. That night Mummu opened his notebook marked "Urgent", just under "coriander", and wrote: "Bigger pot." Then he crossed it out and wrote "More pans."
He'd worked out scaling up versus scaling out without a server in sight.
Where the analogies stop
Here's the honest limitation. These analogies are simplified on purpose. Real setups have more moving parts than a supermarket, and more servers also means more cost to keep an eye on. None of this will make you a cloud architect. What it will do is help you follow what your AI coder is suggesting and why, so when it says "let's put a load balancer in front" you can ask a sensible question back instead of just nodding.
The video version is just over two minutes and ends with a joke I've decided to be proud of: why did the server join a gym? It wanted to scale up. It's episode three of the series on my YouTube channel, and the link is in the first comment.
Next Thursday, we start a fresh set of five.
Until then, open another till before you buy a bigger van.
About the Creator
Gautam Atmakuri
I once spent seven days blindfolded, silent and without food to see what my senses were hiding. I write about that, my baby boy, barefoot running, Tirupati to Auckland life, and the business, marketing and tech I build at AumnI digital.
Enjoyed the story? Support the Creator.
Subscribe for free to receive all their stories in your feed. You could also become a paid subscriber, letting them know you appreciate their work.
Comments
There are no comments for this story
Be the first to respond and start the conversation.