Pagination

how to navigate large collections that expand multiple pages

Introduction

Some API responses may contain a large number of items. To efficiently handle this, the API uses a pagination system based on the page and size parameters or cursors for more efficient pagination.


Link Response Structure

Requests that this list entries in a collection can require pagination to navigate a large number of entries.
When a request support pagination it will respond with a Link http header.

This header includes a link to "next" page of entries, unless on the last page (use this to determine the end of the collection).

Link: <https://api.securityscorecard.io/companies/example.com/...>; rel="next"

In some cases when a collection exceeds a certain size, we'll offer instead a report to asynchronously generate a dump of the entire collection.

An example of this is to Generate a Full Scorecard report to dump all issue findings on a scorecard.



Paginated Response Structure

When a request returns a collection of paginated items, the response structure is as follows:

{ "entries": [ // List of items on the current page ], "total": 1000, // Total number of items in the collection "size": 50, // Number of items per page (default: 50, maximum: 50) "page": 1 // Current page number }

How to Navigate Pages

To iterate through the entire collection, make requests by incrementing the page number until all items have been retrieved.

Requesting the First Page

GET /resource?size=50&page=1

Example with cURL

curl -X GET "https://api.example.com/resource?size=50&page=1" -H "Authorization: Token YOUR_API_KEY"

API Response Example

{ "entries": [ { "id": 1, "name": "Item 1" }, { "id": 2, "name": "Item 2" } ], "total": 1000, "size": 50, "page": 1 }

Requesting the Next Page

To get the next page, increment the page value.

GET /resource?size=50&page=2

Example with cURL

curl -X GET "https://api.securityscorecard.io/resource?size=50&page=2" -H "Authorization: Token YOUR_API_KEY"

Detecting the End of the Collection

Keep paginating until page * size is greater than or equal to total. If entries is empty, it means you have reached the end of the collection.



Cursor-Based Pagination

For large collections, the API may use cursors instead of page and size, allowing for more efficient pagination and preventing skipped items.

Cursor-Based Response Structure

{ "entries": [ { "id": "abc123", "name": "Item 1" }, { "id": "def456", "name": "Item 2" } ], "next_cursor": "ghi789" }

Requesting the First Page with Cursors

GET /resource?limit=50

Example with cURL

curl -X GET "https://api.securityscorecard.io/resource?limit=50" -H "Authorization: Token YOUR_API_KEY"

Requesting the Next Page with Cursors

To get the next page, use the next_cursor returned in the previous response.

GET /resource?limit=50&cursor=ghi789

Example with cURL

curl -X GET "https://api.securityscorecard.io/resource?limit=50&cursor=ghi789" -H "Authorization: Token YOUR_API_KEY"

If next_cursor is not present in the response, it means you have reached the end of the collection.



Handling Large Data Volumes

If the collection exceeds a certain size limit, the API may provide an alternative mechanism, such as generating a full report for asynchronous download.

Generating a Full Report Example

POST /reports/full-dump

Example with cURL

curl -X POST "https://api.securityscorecard.io/reports/full-dump" -H "Authorization: Token YOUR_API_KEY"

Response Example

{ "report_url": "https://api.securityscorecard.io/reports/12345/download", "status": "processing" }

Once generated, you can download the report from report_url.


Best Practices

  • Always use the values returned in the response to determine if more pages are available.
  • Avoid manually constructing pagination URLs. Instead, always use the next_cursor provided in the response for cursor-based pagination or increment the page parameter for page-based pagination. This ensures compatibility with any future API changes and prevents data inconsistencies.
  • Set an appropriate page size (size, default is 50, maximum is 50) to balance performance and resource consumption.
  • Prefer cursor-based pagination for large lists to improve efficiency and prevent inconsistencies.

By following this guide, you can efficiently handle API pagination and optimize data loading in your application.