---
title: "Limit bulk reads and scraping · Agent Wranglers"
url: https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing/limit-bulk-reads
id: AGT-78
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 772
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Limit bulk reads and scraping

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Quotas and pricing](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: AGT-78
- Level: Recommended
- Chapter: [Quotas and pricing](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing.md)
- Rule: Cap how fast one client can page through your data and how much it can read in a day.
- Check: Request 200 pages of a list in one minute from one test client. The later requests are slowed or refused, and a normal reader is not affected.
- How: Count pages read per client and per account. Slow down clients that read more than a person could, such as 60 pages a minute. Use IDs that cannot be guessed in a row, so nobody can walk through every record. Publish what you want read in a feed or sitemap. Page size caps are in [cap the size of every request and answer](https://agentwranglers.se/playbook/api-design/requests-and-responses/request-body-limit.md). Pair it with [rate limits on your APIs](https://agentwranglers.se/playbook/application-security/limits-and-business-logic/rate-limits.md) and a [capped page size](https://agentwranglers.se/playbook/api-design/lists/page-size-limit.md).

## Sources

- [OWASP API4:2023 Unrestricted Resource Consumption](https://api-security.owasp.org/editions/2023/en/0xa4-unrestricted-resource-consumption/)
- [OWASP OAT-011 Scraping](https://github.com/OWASP/www-project-automated-threats-to-web-applications/blob/master/assets/oats/EN/OAT-011_Scraping.md)

## Related rules

- API-93 [Cap the size of every request body](https://agentwranglers.se/playbook/api-design/requests-and-responses/request-body-limit.md) in API design
- APS-49 [Limit how often anyone can call your APIs](https://agentwranglers.se/playbook/application-security/limits-and-business-logic/rate-limits.md) in Application security
- API-22 [Cap the page size](https://agentwranglers.se/playbook/api-design/lists/page-size-limit.md) in API design
- AGT-52 [Decide what each AI crawler may do](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/robots-for-ai-crawlers.md) in Agents as users

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing/limit-bulk-reads">Agent Wranglers agents as users playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Set quotas per agent grant](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing/quota-per-grant.md)
- Next: [Cap what each agent grant may spend](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing/spend-cap-per-grant.md)
- [Quotas and pricing](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing.md)