Pagination

The Data Export API uses cursor-based pagination to handle large datasets supported by all export endpoints

How it works

Cursor-based pagination works by using an opaque token returned in each response to fetch the next page of results:

1. Initial request

   Make your first request with optional time window and page size.

2. Response

   The API returns items and a nextCursor token.

3. Next page

   Use the nextCursor in subsequent requests to fetch the next page.

4. End of results

   When nextCursor is null or empty, you've reached the end.

Parameters

The following parameters control pagination behavior:

start(optional)

integer

The earliest timestamp for records to include in the export in milliseconds since epoch.

end(optional)

integer

The latest timestamp for records to include in the export in milliseconds since epoch.

pageSize

integer

Number of items per page, between 1 and 1000.

Use larger values (500-1000) for bulk exports to reduce API calls.

Default:100

cursor(optional)

string

Pagination cursor returned from the previous response.

Include this to fetch the next page of results. Never modify cursor values.

Examples

The API supports stateful pagination using persistent cursors for reliable periodic exports, and stateless pagination using time windows for simpler one-time exports.

Stateful pagination

You can persist cursors to resume exports exactly where they stopped, preventing data loss. This is the preferred method for ongoing data synchronization, such as daily or hourly exports.

import {Configuration, ExportApi} from '@croct/export';
async function loadCursor(): Promise<string | undefined> {  // Read the previous cursor from a database, file, or cache}
async function saveCursor(cursor: string): Promise<void> {  // Write the current cursor to a database, file, or cache}
async function exportEvents(): Promise<void> {  const api = new ExportApi(    new Configuration({      apiKey: '<API KEY>'    })  );
  let cursor: string | undefined = await loadCursor();
  const pageSize = 100;  let limit = 1000;
  while (limit >= pageSize) {    const {data: {items: events, nextCursor}} = await api.exportEvents({      pageSize: pageSize,      cursor: cursor,    });
    console.log(events);
    cursor = nextCursor;    limit = events.length > 0 ? limit - events.length : 0;  }
  await saveCursor(cursor);}

Stateless Pagination

For simpler one-time exports, you can use time windows without storing cursors. This approach requires no cursor storage but may miss late-arriving or unprocessed events.

import {Configuration, ExportApi} from '@croct/export';
async function exportEvents(): Promise<void> {  const api = new ExportApi(    new Configuration({      apiKey: '<API KEY>'    })  );
  const today = new Date();  today.setHours(0, 0, 0, 0);
  const yesterday = new Date(today.getTime());  yesterday.setDate(today.getDate() - 1);
  const start = Math.floor(yesterday.getTime());  const end = Math.floor(today.getTime());
  let cursor: string | undefined = undefined;  let running = true;
  while (running) {    const {data: {items: events, nextCursor}} = await api.exportEvents({      pageSize: 100,      cursor: cursor,      start: start,      end: end,    });
    console.log(events);
    cursor = nextCursor;    running = events.length > 0;  }}