Part 3 · System design · Data, caching and scale
Caching: cache-aside, TTL, invalidation
Read the cache first; on a miss read the database and fill the cache. On a write, update the database and delete the cache key.
How it works
Reads: get the key from Redis; if it's missing, query the database and set the key with a TTL. Writes: update the database first, then delete the key so the next read refills it. Deleting is safer than setting, because two writers can set the cache in the wrong order and leave a stale value with nothing to fix it. The TTL is the safety net: if an invalidation is ever lost, the stale value still dies when the key expires. When a hot key expires, hundreds of requests miss at the same moment and all hit the database (a stampede). Single-flight makes them share one query.
| hit | about 1 ms (Redis) |
| miss | a database query (often 5 to 50 ms) plus a cache write |
| stale for at most | the TTL, if an invalidation is lost |
| memory | capped by maxmemory with an eviction policy like allkeys-lru |
Reach for it when
- Read heavy data that can be a little stale: profiles, settings, product pages
- Expensive queries and aggregates
- Not for balances or stock counts at checkout: read those from the primary
Implementation (JavaScript, tested)
// TTL cache; `now` is injectable so expiry is testable. In production this is Redis SET key val PX ttl.
class TtlCache {
constructor(ttlMs, now = Date.now) {
this.ttl = ttlMs;
this.now = now;
this.m = new Map();
}
get(key) {
const e = this.m.get(key);
if (!e) return undefined;
if (e.exp <= this.now()) {
this.m.delete(key); // expired counts as a miss
return undefined;
}
return e.val;
}
set(key, val) {
this.m.set(key, { val, exp: this.now() + this.ttl });
}
delete(key) {
this.m.delete(key);
}
}
// Read: cache first, database on a miss, then fill the cache.
function readThrough(cache, db, key) {
const hit = cache.get(key);
if (hit !== undefined) return { val: hit, from: "cache" };
const val = db.get(key);
cache.set(key, val);
return { val, from: "db" };
}
// Write: database first, then DELETE the key. Setting it instead lets two writers race and leave it stale.
function writeAndInvalidate(cache, db, key, val) {
db.set(key, val);
cache.delete(key);
}
// Stampede guard: concurrent misses on one key share a single load.
function singleFlight() {
const inflight = new Map();
return (key, load) => {
if (inflight.has(key)) return inflight.get(key);
const p = Promise.resolve()
.then(load)
.finally(() => inflight.delete(key));
inflight.set(key, p);
return p;
};
}Traps
- Setting the cache on write instead of deleting: a race leaves it stale.
- No TTL: one lost invalidation means stale forever.
- Hot key stampede: single-flight, refresh slightly before expiry, and add random jitter to TTLs so keys don't all expire together.
- Not caching misses: requests for ids that don't exist go straight to the database every time.
In real systems: Redis or Memcached in front of Postgres or MySQL. A CDN is cache-aside for HTTP, with Cache-Control as the TTL. Facebook's memcache paper uses delete on write, plus leases against stampedes.
Say out loud: Cache-aside with a TTL. On writes I update the database, then delete the key rather than set it, so racing writers can't leave it stale. Hot keys get single-flight, so one expiry doesn't become a thousand queries.