Should we cache database queries or full API responses?

API responses give the largest latency win because they cache the entire computation (query + serialization + business logic). Database query caching is a finer-grained fallback for shared subqueries. Both are valid; response caching is the higher-leverage first move for read-heavy workloads.

How do we cache authenticated content?

Include the user identity in the cache key (or use Vary headers at the CDN layer). Never cache authenticated responses without user-scoping — one of the most common security incidents is a public CDN serving one user's data to another.

When should we not cache?

Real-time data (live prices, inventory that must be accurate to the second), personalized data with low reuse (cache hit rate below ~30% means you're paying complexity cost for little benefit), and anything where staleness has legal or safety consequences. Not everything benefits from caching; profile first, cache second.

Caching Strategy: Where to Cache, What to Invalidate

Caching stores the result of an expensive computation or query so subsequent requests can reuse it.

Caching Strategy: The Optimization That Creates Its Own Category of Bug

Caching is the practice of storing the result of an expensive operation so future identical requests can reuse it instead of recomputing. Phil Karlton's line — 'there are only two hard things in computer science: cache invalidation and naming things' — endures because caching adds a shadow layer of state that must be kept coherent with the source of truth. Every cache buys latency and throughput at the cost of complexity, staleness risk, and a new category of bug where the code is correct but the cached data is not.

Where caching lives

Layered from client to database. (a) Browser cache — HTTP Cache-Control headers, service workers, IndexedDB. Cheapest, invalidation via versioned URLs. (b) CDN — cached at edge locations for global reach, invalidation via purge APIs. (c) Reverse proxy — Varnish, Cloudflare, Fastly at the application boundary. (d) Application cache — Redis, Memcached, in-process (rare, hard to invalidate across instances). (e) Database cache — buffer pool, query plan cache, materialized views. Each layer serves different access patterns; caching at the wrong layer often means you're solving the wrong problem.

Invalidation strategies

(1) Time-based (TTL) — simplest, accepts staleness up to the TTL. Works well for data that changes on human timescales and can tolerate lag (top-N lists, pricing tables refreshed hourly). (2) Event-based — invalidate on write. Correct in theory, tricky in practice because every write path must know every cache key affected. (3) Versioned keys — cache key includes a version stamp; writing bumps the version, effectively creating a new cache entry and orphaning the old. Simple to reason about, wastes some memory on stale entries. Most systems combine short TTLs with event-based invalidation for known-critical writes.

The thundering herd problem

When a cache entry expires under high load, many requests simultaneously miss and try to regenerate the value — hammering the origin. Solutions: (a) probabilistic early recomputation — refresh the cache slightly before expiry with a probability that increases as expiry approaches. (b) request coalescing / singleflight — only one request regenerates, others wait for its result. (c) stale-while-revalidate — serve the stale value while a background job refreshes. Every serious cache library implements at least one of these; not using them is how a single popular URL takes down the origin during a traffic spike.

Cache-aside vs. read-through vs. write-through

Cache-aside (most common): application checks cache first, falls back to database, populates cache. Application controls the cache; database doesn't know it exists. Read-through: cache library owns the fetch — simpler application code, less flexibility. Write-through: writes go through the cache to the database — cache is always consistent but writes are slower. Write-behind: writes go to the cache and are asynchronously flushed to the database — fastest, most dangerous, appropriate only for tolerable-loss workloads (analytics, session state).

Debugging cache-related incidents

When production returns wrong data, cache is a common suspect and a hard one to diagnose. Instrumentation that pays off: log cache hits/misses per key pattern; expose a debug header (X-Cache: HIT/MISS/BYPASS) in responses; support a request-time bypass parameter for internal debugging; run a small percentage of traffic uncached in staging for parity checks. Caches that can't be inspected or bypassed become black boxes during incidents, and that's when you find out how much you were relying on them.

Frequently asked questions

Should we cache database queries or full API responses?: API responses give the largest latency win because they cache the entire computation (query + serialization + business logic). Database query caching is a finer-grained fallback for shared subqueries. Both are valid; response caching is the higher-leverage first move for read-heavy workloads.
How do we cache authenticated content?: Include the user identity in the cache key (or use Vary headers at the CDN layer). Never cache authenticated responses without user-scoping — one of the most common security incidents is a public CDN serving one user's data to another.
When should we not cache?: Real-time data (live prices, inventory that must be accurate to the second), personalized data with low reuse (cache hit rate below ~30% means you're paying complexity cost for little benefit), and anything where staleness has legal or safety consequences. Not everything benefits from caching; profile first, cache second.