Rotate secrets automatically

How to do it

Turn on the store's rotation, such as rotation functions in AWS Secrets Manager or dynamic secrets in Vault. Keep the old and new value valid together while the switch happens. The app caches a secret for 1 hour at most. It fetches it again when a call fails with an authentication error. Rotate at once after a leak, with the same job. Where the store cannot rotate a secret, rotate it by hand within the same 90 days, as keys with the least power requires.

Example

# Prints every secret that does not rotate or rotated over 90 days ago. Expect nothing.
aws secretsmanager list-secrets --output json | jq -r --arg limit "$(date -u -d '-90 days' +%FT%T)" \
  '.SecretList[] | select(.RotationEnabled != true or (.LastRotatedDate // "") < $limit) | .Name'

import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

const CACHE_MS = 60 * 60 * 1000;
const store = new SecretsManagerClient({});
const cache = new Map();

export async function getSecret(name, { refresh = false } = {}) {
  const cached = cache.get(name);
  if (!refresh && cached && Date.now() - cached.fetchedAt < CACHE_MS) return cached.value;
  const { SecretString } = await store.send(new GetSecretValueCommand({ SecretId: name }));
  cache.set(name, { value: SecretString, fetchedAt: Date.now() });
  return SecretString;
}
// On an authentication error: getSecret(name, { refresh: true }) and retry once.

import time
import boto3

CACHE_SECONDS = 3600
store = boto3.client("secretsmanager")
cache = {}

def get_secret(name, refresh=False):
    cached = cache.get(name)
    if not refresh and cached and time.monotonic() - cached[1] < CACHE_SECONDS:
        return cached[0]
    value = store.get_secret_value(SecretId=name)["SecretString"]
    cache[name] = (value, time.monotonic())
    return value

How to check

The query below lists no secret without rotation or with a last rotation over 90 days ago. Documented secrets rotated by hand within 90 days are the only exception. Trigger a rotation on staging, and the app keeps serving without a restart.