Keep the app working when the cache is gone

How to do it

Give cache calls a short time limit, such as 50 ms, far below the request's own limit from Set timeouts on every call. Catch cache errors, log them and fall back to the source of truth. An empty cache after a restart sends every read to the database at once. Test that the database survives this, or refuse the excess as Turn work away before overload does. The Google SRE book warns that a service that copes only with a warm cache is at higher risk of outage.

Examples

JavaScript example

const CACHE_TIMEOUT_MS = 50;

export async function readThrough(cache, key, load) {
  try {
    const hit = await Promise.race([
      cache.get(key),
      new Promise((_, reject) => setTimeout(() => reject(new Error('cache timeout')), CACHE_TIMEOUT_MS)),
    ]);
    if (hit !== null) return JSON.parse(hit);
  } catch (error) {
    log.warn({ key, error: error.message }, 'cache unavailable, reading source');
  }
  return load();
}

Python example

import json
import logging

import redis

CACHE_TIMEOUT_SECONDS = 0.05
cache = redis.Redis(host="cache.internal.example.com", socket_timeout=CACHE_TIMEOUT_SECONDS)

def read_through(key: str, load):
    try:
        hit = cache.get(key)
        if hit is not None:
            return json.loads(hit)
    except redis.RedisError as error:
        logging.warning("cache unavailable, reading source: %s", error)
    return load()

How to check

Stop the cache server on staging while the load test runs at the planned peak. The error rate stays under 1%. After the cache returns, response times are back to normal within 5 minutes.

Sources

• Google SRE book, Addressing Cascading Failures

Related rules

• Set timeouts on every call in API design
• Turn work away before overload in Operations