Agent Wranglers

DAT-64Recommended

Cap the size of every search

Reject search text over a fixed length, such as 200 characters, and return a fixed number of hits per page. Stop deep paging at a fixed depth, such as 1,000 hits.

How to do it

Long input makes many terms, and each term adds index reads. Ranking reads the tsvector of every match, so a common word on a large table is slow. Check the length on the server before the query runs, and answer 400 with the limit. Ask for one hit more than the page size to know whether a next page exists, instead of counting every match. Page size caps come from Set a default page size and a maximum. Time limits come from Set time limits on every query. Request limits for search come from Limit requests per client.

Examples

JavaScript example

const MAX_QUERY_CHARACTERS = 200;
const MAX_SEARCH_DEPTH = 1000;
if (query.length > MAX_QUERY_CHARACTERS) return problem(400, `Search text is limited to ${MAX_QUERY_CHARACTERS} characters`);
if (offset + pageSize > MAX_SEARCH_DEPTH) return problem(400, `Search results stop at ${MAX_SEARCH_DEPTH} hits`);

Python example

MAX_QUERY_CHARACTERS = 200
MAX_SEARCH_DEPTH = 1000
if len(query) > MAX_QUERY_CHARACTERS:
    return problem(400, f"Search text is limited to {MAX_QUERY_CHARACTERS} characters")
if offset + page_size > MAX_SEARCH_DEPTH:
    return problem(400, f"Search results stop at {MAX_SEARCH_DEPTH} hits")

How to check

Send a search of 201 characters. The answer is 400 and names the limit. Ask for page 51 at 20 hits a page. The answer is 400 or an empty page, within the query time limit.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/data/search/search-limits">Agent Wranglers data playbook</a>. In code you write, add a comment next to it with the same link.