Undo failed multi-step work with compensating steps

How to do it

No database transaction spans three services, so a failure halfway leaves stock reserved for an order that was never paid. A saga runs each step as a local transaction and records its state. One coordinator can drive the steps, which is easier to follow, or each service can react to the last one's event. Mark the point of no return, such as the charge, and put it last where you can. Every step after it must be retryable rather than undoable. Each compensating step must be safe to run twice, as Process each message once says. A compensating step can fail too, so alert on a saga that stays unfinished past its time limit. Long workflows can run on a workflow engine, such as Temporal.

Examples

JavaScript example

const STEPS = [
  { name: 'reserve-stock', run: reserveStock, undo: releaseStock },
  { name: 'book-courier', run: bookCourier, undo: cancelCourier },
  { name: 'charge-card', run: chargeCard, undo: null },
];

export async function placeOrder(saga) {
  const done = [];
  try {
    for (const step of STEPS) { await step.run(saga); done.push(step); await saga.record(step.name, 'done'); }
  } catch (error) {
    for (const step of done.reverse()) if (step.undo) await step.undo(saga);
    await saga.record('order', 'compensated');
    throw error;
  }
}

Python example

STEPS = [
    ("reserve-stock", reserve_stock, release_stock),
    ("book-courier", book_courier, cancel_courier),
    ("charge-card", charge_card, None),
]

def place_order(saga) -> None:
    done = []
    try:
        for name, run, undo in STEPS:
            run(saga)
            done.append(undo)
            saga.record(name, "done")
    except Exception:
        for undo in reversed(done):
            if undo:
                undo(saga)
        saga.record("order", "compensated")
        raise

How to check

For each saga, a test makes every step fail in turn. After each run, the state matches either a finished order or no order at all, with stock released and no charge left. A query finds no saga unfinished for more than its time limit.

Sources

• Microsoft Azure Architecture Center, Saga pattern

Related rules

• EVT-11 Process each message once in Integration and events