Cap serverless concurrency and measure cold starts

How to do it

Serverless platforms start a new copy for each request that finds no free one, so a spike can start hundreds of copies in seconds. Each opens its own database connections, and the database runs out first. Set the function's own concurrency limit, such as reserved concurrency on AWS Lambda or maximum instances on Cloud Run. Put a connection pooler or a data API between the functions and the database. A new copy loads its code and opens connections before it answers, which is the cold start. Keep the package small and open connections outside the handler, so later requests reuse them. Warm copies, such as provisioned concurrency or minimum instances, cost money while idle, so add them only where the measured cold start breaks the objective. The budget itself is in Fit the connection budget to the autoscaling maximum.

Examples

Terraform example

resource "aws_lambda_function" "checkout" {
  function_name                  = "shop-checkout"
  reserved_concurrent_executions = 50
  # other settings left out
}

JavaScript example

// Open the pool once per copy, outside the handler, so warm requests reuse it
import pg from 'pg';
const pool = new pg.Pool({ max: 2, connectionString: process.env.DATABASE_URL });

export const handler = async event => {
  const { rows } = await pool.query('SELECT id, total FROM orders WHERE id = $1', [event.orderId]);
  return rows[0];
};

Python example

import os
import psycopg_pool

pool = psycopg_pool.ConnectionPool(os.environ["DATABASE_URL"], max_size=2)

def handler(event, context):
    with pool.connection() as conn:
        return conn.execute("SELECT id, total FROM orders WHERE id = %s", (event["order_id"],)).fetchone()

How to check

Every function in the infrastructure code sets a maximum concurrency or instance count. That maximum times connections per copy stays within the connection budget. A report lists the 95th percentile cold start per function. Each function with warm copies has a note naming the objective a cold start broke.

Sources

• AWS Lambda, Configuring reserved concurrency
• Cloud Run, Set maximum instances

Related rules

• SCL-23 Fit the connection budget to the autoscaling maximum in Scaling