Find and contain noisy neighbours

How to do it

In shared tables, one tenant's large report slows every other tenant's pages. Tag each query with the tenant, in a SQL comment or the connection's application_name. The slow query log and query statistics then group by tenant. Cap each tenant's running jobs, as Give each customer a fair share shows. Set the query time limit from Set time limits on every query per role or per tenant tier. A tenant that stays above a quarter of the load moves to its own database. That is the mixed model in Choose the isolation model before the first customer.

Examples

JavaScript example

export const tagQuery = (tenantId, sql) => `/* tenant_id=${Number(tenantId)} */ ${sql}`;

const rows = await db.any(tagQuery(req.tenant.id, 'SELECT id, total FROM invoices WHERE due_at < now()'));

Python example

def tag_query(tenant_id: int, sql: str) -> str:
    return f"/* tenant_id={int(tenant_id)} */ {sql}"

rows = conn.execute(tag_query(tenant.id, "SELECT id, total FROM invoices WHERE due_at < now()")).fetchall()

How to check

A dashboard lists the 10 tenants with the most database time over the last 24 hours. An alert fires when one tenant passes 25% of the total for an hour. On staging, one tenant runs a heavy report, and another tenant's p95 page time stays within 20% of its baseline.

Sources

• AWS SaaS Lens, noisy neighbour

Related rules

• DAT-70 Put the tenant on every log line and trace in Data and databases
• SCL-31 Give each customer a fair share in Scaling
• DAT-37 Set time limits on every query in Data and databases
• DAT-67 Choose the isolation model before the first customer in Data and databases