---
title: "Cap serverless concurrency and measure · Agent Wranglers"
url: https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/serverless-concurrency
id: SCL-44
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1082
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Cap serverless concurrency and measure cold starts

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: SCL-44
- Level: Recommended
- Chapter: [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling.md)
- Rule: Give every function a maximum number of copies running at once that fits the database connection budget. Measure cold start time, and keep warm copies only for routes whose speed objective a cold start breaks.
- Check: Every function in the infrastructure code sets a maximum concurrency or instance count. That maximum times connections per copy stays within the connection budget. A report lists the 95th percentile cold start per function. Each function with warm copies has a note naming the objective a cold start broke.
- How: Serverless platforms start a new copy for each request that finds no free one, so a spike can start hundreds of copies in seconds. Each opens its own database connections, and the database runs out first. Set the function's own concurrency limit, such as reserved concurrency on AWS Lambda or maximum instances on Cloud Run. Put a connection pooler or a data API between the functions and the database. A new copy loads its code and opens connections before it answers, which is the cold start. Keep the package small and open connections outside the handler, so later requests reuse them. Warm copies, such as provisioned concurrency or minimum instances, cost money while idle, so add them only where the measured cold start breaks the objective. The budget itself is in [Fit the connection budget to the autoscaling maximum](https://agentwranglers.se/playbook/scaling/growing-the-database/connection-budget.md).

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### Terraform example

```hcl
resource "aws_lambda_function" "checkout" {
  function_name                  = "shop-checkout"
  reserved_concurrent_executions = 50
  # other settings left out
}
```

### JavaScript example

```js
// Open the pool once per copy, outside the handler, so warm requests reuse it
import pg from 'pg';
const pool = new pg.Pool({ max: 2, connectionString: process.env.DATABASE_URL });

export const handler = async event => {
  const { rows } = await pool.query('SELECT id, total FROM orders WHERE id = $1', [event.orderId]);
  return rows[0];
};
```

### Python example

```python
import os
import psycopg_pool

pool = psycopg_pool.ConnectionPool(os.environ["DATABASE_URL"], max_size=2)

def handler(event, context):
    with pool.connection() as conn:
        return conn.execute("SELECT id, total FROM orders WHERE id = %s", (event["order_id"],)).fetchone()
```

## Sources

- [AWS Lambda, Configuring reserved concurrency](https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html)
- [Cloud Run, Set maximum instances](https://cloud.google.com/run/docs/configuring/max-instances)

## Related rules

- SCL-23 [Fit the connection budget to the autoscaling maximum](https://agentwranglers.se/playbook/scaling/growing-the-database/connection-budget.md) in Scaling

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/serverless-concurrency">Agent Wranglers scaling playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Turn off optional work under load](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/degrade-under-load.md)
- Next: [Queue model calls within the provider's rate limits](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/model-rate-limits.md)
- [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling.md)