You can throw infinite servers at a program and still watch it fall over, because some scale problems live inside the code. The good news: the primitives that fix them were invented decades ago and have never been beaten.
The series so far has scaled the system from the outside — distributing the work and the delivery. This post goes the other way: down into a single program. Because the difference between code that handles one user and code that handles a million is rarely the hardware. It is whether you used the old, boring primitives that scale — or the obvious ones that don't.
Five primitives older than most of the industry
Every one of these predates the cloud, predates the web, in some cases predates the people using them. They are still the answer.
1. The index — turn a search into a lookup
Scanning every row to find one is the original sin of scale. The index — a sorted structure that turns a linear scan into a logarithmic lookup — is from the 1970s relational era and is still the single highest-leverage line of code you can add to a slow system.
-- O(n): full scan, fine at 1k rows, fatal at 10M
SELECT * FROM orders WHERE customer_id = 42;
-- one old idea, and the same query is O(log n)
CREATE INDEX idx_orders_customer ON orders(customer_id);2. Batching — pay the fixed cost once
Every operation has a fixed overhead — a network round trip, a transaction, a syscall. Doing it per-item is how a program dies under load. Batching, the mainframe's native instinct, amortises that fixed cost across many items. It is the difference between a thousand round trips and one.
// the N+1 killer — one trip per item
for (const id of ids) await db.insert(row(id));
// pioneer instinct: batch the fixed cost away
await db.insertMany(ids.map(row)); // one trip3. Streaming — never hold the whole thing in memory
Load a 5 GB file to process it and you scale exactly until the file is bigger than RAM. Process it as a stream — one chunk at a time, the Unix-pipe instinct again — and memory stops mattering. The program that streams handles inputs it could never fit.
4. The cache — don't compute what you already know
The oldest optimisation there is: remember the answer. A cache trades a little memory and some staleness for an enormous drop in repeated work. The hard part was never the idea — it is invalidation, and that is a discipline, not a technology.
5. Back-pressure — let the system say 'slow down'
A system with no way to refuse work doesn't scale; it collapses. Back-pressure — queues with limits, rate limits, bounded buffers — is a 1960s flow-control idea that keeps a program degrading gracefully instead of falling over. Saying 'not yet' is a feature.
The hierarchy that actually matters
Before you reach for more machines, walk down this list — it is ordered by leverage, and the cheap wins are all at the top and all decades old.
- Fix the algorithm — an O(n²) loop beats no hardware. Big-O is from the 1960s and still decides everything.
- Add the index — turn scans into lookups.
- Batch the I/O — amortise the fixed cost.
- Cache the repeats — stop recomputing known answers.
- Then, and only then, distribute — scale out the work that is already efficient.
How we hold the line in the studio
Our rule is simple: a feature is not done when it works, it is done when it works at the size we expect to grow into. That means the index goes in with the query, the loop that hits the network gets batched, and anything that reads a file reads it as a stream. None of it is clever. All of it is old. All of it is why small systems we build keep running long after the demo.
Most scaling is just refusing to do the same expensive thing twice.
— Studio principle
Efficient code that distributes and delivers well is most of the battle. The last piece is time: building so the thing still runs in five years. That is part five — built to last.
If your app got slow as it got popular, the fix is usually five old primitives away — not a bigger server bill.
Let us look at itQuestions, answered
- Isn't scaling just a matter of adding servers?
- Only after the code is efficient. Adding machines to an O(n²) algorithm or an N+1 query multiplies the cost without fixing the cause. Indexing, batching, streaming and caching are the cheap wins that come first — infrastructure scale comes last.
- Why do these old primitives still beat newer approaches?
- Because they target fixed physics — round-trip latency, memory limits, comparison counts — that haven't changed. An index turns a scan into a lookup for the same reason it did in 1975. The constraints are durable, so the solutions are too.
The stack we actually use
The tools below run our own systems and the ones we build for clients. A few are affiliate links — they support The Signal at no cost to you.
Some links on this site are affiliate links. If you click through and sign up or buy, J Supreme Tech may earn a commission at no extra cost to you. We only recommend tools we actually use to build and run our own systems. Full disclosure.