Agent Wranglers

Queries and indexes Recommended

Page through long lists by key

For long lists, page by the last key seen instead of OFFSET. Offsets get slower with every page and skip or repeat rows when data changes.

How to do it

Sort by a unique key or a unique pair, index those columns in that order, and ask for rows after the last one shown. MySQL cannot use an index for the row comparison below, so write it there as created_at < $1 OR (created_at = $1 AND id < $2). Cap the page size on the server, as the resource limits rule says. The API playbook's cursor rule shows how clients receive the key.

Example

SELECT id, created_at, total_cents FROM orders
WHERE (created_at, id) < ($1, $2)
ORDER BY created_at DESC, id DESC
LIMIT 50;

How to check

Follow the keys to a page 50,000 rows into a large list. It answers as fast as page 1, and its plan shows an index range scan.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers data playbook: https://agentwranglers.se/playbook/data/queries-and-indexes/keyset-pagination.