Spread the reads of one hot cache key

How to do it

A cache cluster spreads keys across servers, but one key always lives on one server. A price list or a feature flag read on every request sends all that traffic to one cache server. It saturates while the others idle. Count reads per key with the cache's own tools, such as redis-cli --hotkeys with an LFU eviction policy, or by sampling in the app. Keep a small local copy in each app process for 1 to 5 seconds. Each process then asks the cache once per lifetime. When even that is too much, write the value under several keys, such as price-list:0 to price-list:7, and let each reader pick one at random. Hot rows in the database follow Spread writes that all land on one row. A rebuild of a hot key follows Let one caller rebuild an expired entry.

Examples

JavaScript example

const LOCAL_TTL_MS = 2000;
const localCopies = new Map();

export async function readHot(cache, key) {
  const local = localCopies.get(key);
  if (local && local.expiresAt > Date.now()) return local.value;
  const value = await cache.get(key);
  localCopies.set(key, { value, expiresAt: Date.now() + LOCAL_TTL_MS });
  return value;
}

Python example

import time

LOCAL_TTL_SECONDS = 2.0
local_copies: dict[str, tuple[float, object]] = {}

def read_hot(cache, key: str):
    expires_at, value = local_copies.get(key, (0.0, None))
    if expires_at > time.monotonic():
        return value
    value = cache.get(key)
    local_copies[key] = (time.monotonic() + LOCAL_TTL_SECONDS, value)
    return value

How to check

Under the load test at the planned peak, the busiest cache server's CPU is at most 2 times the average of the others. No single key takes more than 10% of the reads that reach the cache.

Sources

• Redis key eviction and LFU

Related rules

• SCL-40 Spread writes that all land on one row in Scaling
• SCL-13 Let one caller rebuild an expired entry in Scaling