Why Systems Slow Down and What Smart Caching Teaches Us About Scalability

Modern applications usually start fast. But as traffic grows, so does the load on the backend — and somewhere along the way, things slow down. Often, it’s not bad code or poor DB design — it’s the volume of repeated reads hitting your database like a DDoS. Caching becomes the first (and sometimes only) line of defense. But caching isn’t just about speed. It’s about trade-offs — consistency, durability, and failure recovery.

So…what exactly is a cache?

A cache is memory that stores frequently accessed data, so you don’t have to hit your database or expensive downstream systems every time.

But in real systems, a cache is not just a faster version of your database. It’s a separate layer that has its own lifecycle, consistency rules, and edge cases.

Let’s start with the typical, unoptimized requaest flow:

Unoptimized data flow between the user, frontend, backend, and database.

Repeat this for every user, every second, and your DB will cry for help.

Choosing the Right Cache Strategy

1. Local (In-Process) Cache

With local caching, each server instance stores data in its memory. It’s blazingly fast — there are no network hops, just RAM access.

But this comes at a cost. Since every instance has its copy of the cache, data updates don’t automatically sync across them. This can lead to inconsistencies.

In setups with multiple services or containers, this quickly becomes a fan-out problem.

App instances with separate local caches for each instance.

To make this work reliably, you'd need sharding, coordination, and sometimes even replication logic — adding operational complexity.

2. Global (Centralized) Cache

This is where tools like Redis or Memcached shine. Instead of each node caching data independently, all instances talk to a shared in-memory store.

Apps using Redis as a centralized cache with PostgreSQL as the database.

Now, if a value is updated, it’s immediately visible to all instances — solving the consistency problem. The downside? Every cache access is a network call. Still fast, but not as instant as a local memory lookup.

3. Distributed Cache with Sharding + Replication

This setup partitions the cache across nodes (sharding), and replicates data across machines for fault tolerance.

Distributed cache with client, coordinator, and sharded cache nodes .

To maintain consistency, you typically use quorum logic:

If total nodes = 3, and you write to 2 (W=2), then you must read from at least 2 (R=2) to be safe, because R + W > N.

Handling Writes: Where Things Start Getting Real

➔ Write-Through Cache

Every write goes to both the cache and the database, synchronously.

Write-Through Cache

Reliable and consistent, but adds latency.

➔ Write-Back (Write-Behind) Cache

Here, the write is stored in cache and acknowledged immediately. The database is updated later, often asynchronously.

Write-Back Cache

Fast, but if the cache crashes, you lose data unless you persist elsewhere.

➔ Write-Around Cache Skip the cache entirely for writes. Cache only comes into play during reads.

Write-Around Cache

Good for cold data, but every first read is a miss.

➔ Cache-Aside (Lazy Loading)

App explicitly manages reads and writes. On a cache miss, fetch from the database and then write to the cache. On writes, update DB and invalidate cache.

It gives full control but demands discipline.

Ensuring Consistency with Quorum Reads

If you are using a distributed cache, ensure R + W > N to read from at least one up-to-date node. Otherwise, you might serve stale data from a node that hasn’t received the latest write.

Cache Invalidation: The Real Headache

Common patterns:

TTL: Keys expire after a fixed time.

Manual Invalidation: Delete the cache entry after DB write.

Pub/Sub: Broadcast cache bust messages.

Versioned Keys: Use versioning in keys to force reads to new data.

Eviction Strategies: When Memory Runs Out

Common Strategies:

LRU (Least Recently Used)

LFU (Least Frequently Used)

Segmented LRU (used in Memcached)

Cache lifecycle: key eviction from hot region and reuse in cold region

Choose based on your app's access patterns.

Summary: Pick Based on Trade-off

Write-through caching provides strong consistency because data is written to both the cache and the database at the same time. However, write performance is slower since database latency directly affects every write operation. This strategy is commonly used for critical systems such as user profiles, settings, and payment transactions where consistency is important.

Write-back caching focuses on speed by first writing data only to the cache and updating the database later in the background. While this approach offers very fast write performance, it carries the risk of data loss if the cache crashes before the database is updated. It is mainly suitable for use cases like logs, counters, and analytics systems where slight delays in consistency are acceptable.

Write-around caching skips writing data to the cache during updates and writes directly to the database instead. This reduces unnecessary cache usage but can lead to cache misses when recently updated data is requested. It provides eventual consistency with medium performance and is often used for product catalogs or metadata systems where read patterns are less predictable.

Cache-aside caching gives developers full control over when data should be loaded into or removed from the cache. It is highly flexible and works well for API-driven applications, GraphQL systems, and mixed read workloads. However, developers must carefully handle cache invalidation to avoid stale data issues.

Ask yourself:

Is the data read-heavy or write-heavy?

Can you tolerate eventual consistency?

How will you handle invalidation?

What’s your eviction strategy under load?

Final Thoughts

Caching is not just a performance trick — it’s a system design decision. Used right, it can speed up systems by 10x. Used incorrectly, it silently causes data bugs that surface only in production.

Plan your cache like you're planning your database. Design for failure. Test for staleness. Let’s build systems that scale and stay correct.