Let one caller rebuild an expired entry

How to do it

Without this, every request that misses at the same moment runs the same slow query. The database then takes the full load just as the cache empties. Within one process, share the pending promise or future for each key. Across copies, take a short lock per key as Keep locks and counters in a shared store shows, and serve the stale value meanwhile. Add up to 10% random time to each expiry, so entries written together do not expire together.

Examples

JavaScript example

const pending = new Map();
const JITTER_SHARE = 0.1;

export async function cached(cache, key, ttlSeconds, load) {
  const hit = await cache.get(key);
  if (hit !== null) return JSON.parse(hit);
  if (!pending.has(key)) {
    const ttl = Math.round(ttlSeconds * (1 + Math.random() * JITTER_SHARE));
    pending.set(key, load()
      .then(async value => { await cache.set(key, JSON.stringify(value), 'EX', ttl); return value; })
      .finally(() => pending.delete(key)));
  }
  return pending.get(key);
}

Python example

import asyncio
import json
import random

JITTER_SHARE = 0.1
_pending: dict[str, asyncio.Future] = {}

async def cached(cache, key: str, ttl_seconds: int, load):
    hit = await cache.get(key)
    if hit is not None:
        return json.loads(hit)
    if key not in _pending:
        _pending[key] = asyncio.ensure_future(_rebuild(cache, key, ttl_seconds, load))
    return await _pending[key]

async def _rebuild(cache, key, ttl_seconds, load):
    try:
        value = await load()
        ttl = round(ttl_seconds * (1 + random.random() * JITTER_SHARE))
        await cache.set(key, json.dumps(value), ex=ttl)
        return value
    finally:
        _pending.pop(key, None)

How to check

In a test, expire a hot key while 500 requests read it at once. The source is called at most once per app copy, and no request fails.

Related rules

• Keep locks and counters in a shared store in Scaling