Pagination
Pagination is the process of dividing a large collection of API results into smaller groups that can be requested separately.
Instead of returning every record in one response, the API returns one page or segment at a time.
What It Means in Practice
Imagine an account contains:
12,000 customers.
Returning all 12,000 records in every request could be slow and expensive.
The API might instead return:
100 customers
plus information describing how to retrieve the next group.
Common Pagination Approaches
APIs may paginate using:
- Page numbers
- Offsets
- Cursors
- Continuation tokens
- Links to the next page
The specific approach belongs to the API contract.
GitHub's REST documentation explains how large result sets are divided into separate responses and exposes navigation information for subsequent pages. GitHub REST API pagination documentation
Its GraphQL API uses cursor-based pagination for larger collections. GitHub GraphQL pagination guide
Why It Matters
An integration that retrieves only the first response may silently miss most of the data.
For example:
API says 30 customers.
The business actually has:
2,400 customers.
The integration may appear to work while remaining incomplete.
Pagination and Synchronization
Pagination becomes particularly important for:
- Data Synchronization
- Imports
- Product catalogues
- Analytics exports
- Content migrations
- Account reconciliation
The integration needs to continue requesting pages until it reaches the intended end of the dataset.
Common Problems
Common problems include:
- Reading only the first page
- Hardcoding page counts
- Records changing while pagination is underway
- Duplicate records between pages
- Rate Limits
- Assuming every API uses the same pagination method
- Very large page sizes creating slow responses
How to Improve
Follow the pagination mechanism provided by the API.
Do not manually construct pagination behaviour if the API supplies continuation URLs or cursor values.
For large synchronization jobs, record progress so a failed run does not necessarily need to restart from the beginning.