# Pagination

Learn how to paginate through large datasets.

The Data Export API uses cursor-based pagination to handle large datasets supported by all export endpoints

## How it works

Cursor-based pagination works by using an opaque token returned in each response to fetch the next page of results:

1. **Initial request**

   Make your first request with optional time window and page size.

2. **Response**

   The API returns items and a `nextCursor` token.

3. **Next page**

   Use the `nextCursor` in subsequent requests to fetch the next page.

4. **End of results**

   When `nextCursor` is an empty string, you've reached the end.

## Parameters

The following parameters control pagination behavior:

- `start`: `integer` (optional)

  The earliest timestamp for records to include in the export in milliseconds since epoch.

- `end`: `integer` (optional)

  The latest timestamp for records to include in the export in milliseconds since epoch.

- `pageSize`: `integer` (optional) (default: 100)

  Number of items per page, between 1 and 1000.

  Use larger values (500-1000) for bulk exports to reduce API calls.

- `cursor`: `string` (optional)

  Pagination cursor returned from the previous response.

  Include this to fetch the next page of results. Never modify cursor values.

## Examples

The API supports stateful pagination using persistent cursors for reliable periodic exports, and stateless pagination using time windows for simpler one-time exports.

### Stateful pagination

You can persist cursors to resume exports exactly where they stopped, preventing data loss. This is the preferred method for ongoing data synchronization, such as daily or hourly exports.

**Stateful pagination with cursor persistence**

**TypeScript**

```ts
import {Configuration, ExportApi} from '@croct/export';

async function loadCursor(): Promise<string | undefined> {
  // Read the previous cursor from a database, file, or cache
}

async function saveCursor(cursor: string): Promise<void> {
  // Write the current cursor to a database, file, or cache
}

async function exportEvents(): Promise<void> {
  const api = new ExportApi(
    new Configuration({
      apiKey: '<API KEY>'
    })
  );

  let cursor: string | undefined = await loadCursor();

  const pageSize = 100;
  let limit = 1000;

  while (limit >= pageSize) {
    const {data: {items: events, nextCursor}} = await api.exportEvents({
      pageSize: pageSize,
      cursor: cursor,
    });

    console.log(events);

    cursor = nextCursor;
    limit = events.length > 0 ? limit - events.length : 0;
  }

  await saveCursor(cursor);
}
```

**JavaScript**

```js
import {Configuration, ExportApi} from '@croct/export';

async function loadCursor() {
  // Read the previous cursor from a database, file, or cache
}

async function saveCursor(cursor) {
  // Write the current cursor to a database, file, or cache
}

async function exportEvents() {
  const api = new ExportApi(
    new Configuration({
      apiKey: '<API KEY>'
    })
  );

  let cursor = await loadCursor();

  const pageSize = 100;
  let limit = 1000;

  while (limit >= pageSize) {
    const {data: {items: events, nextCursor}} = await api.exportEvents({
      pageSize: pageSize,
      cursor: cursor,
    });

    console.log(events);

    cursor = nextCursor;
    limit = events.length > 0 ? limit - events.length : 0;
  }

  await saveCursor(cursor);
}
```

**Java**

```java
import com.croct.client.export.ApiClient;
import com.croct.client.export.ApiException;
import com.croct.client.export.api.ExportApi;
import com.croct.client.export.model.Event;
import com.croct.client.export.model.EventResponse;
import java.util.List;

public class StatefulPagination {
  public static void main(String[] args) throws ApiException {
    final ApiClient client = new ApiClient();
    client.setApiKey("<API KEY>");

    final ExportApi api = new ExportApi(client);

    String cursor = loadCursor();

    final int pageSize = 100;
    int limit = 1000;

    while (limit >= pageSize) {
      final EventResponse response = api.exportEvents(
        null,
        null,
        pageSize,
        cursor,
        null
      );

      final List<Event> events = response.getItems();

      System.out.println(events);

      cursor = response.getNextCursor();
      limit = events.isEmpty() ? 0 : limit - events.size();
    }

    saveCursor(cursor);
  }

  private static String loadCursor() {
    // Read the previous cursor from a database, file, or cache
    return null;
  }

  private static void saveCursor(final String cursor) {
    // Write the current cursor to a database, file, or cache
  }
}
```

### Stateless Pagination

For simpler one-time exports, you can use time windows without storing cursors. This approach requires no cursor storage but may miss late-arriving or unprocessed events.

**Time-based pagination**

**TypeScript**

```ts
import {Configuration, ExportApi} from '@croct/export';

async function exportEvents(): Promise<void> {
  const api = new ExportApi(
    new Configuration({
      apiKey: '<API KEY>'
    })
  );

  const today = new Date();
  today.setHours(0, 0, 0, 0);

  const yesterday = new Date(today.getTime());
  yesterday.setDate(today.getDate() - 1);

  const start = Math.floor(yesterday.getTime());
  const end = Math.floor(today.getTime());

  let cursor: string | undefined = undefined;
  let running = true;

  while (running) {
    const {data: {items: events, nextCursor}} = await api.exportEvents({
      pageSize: 100,
      cursor: cursor,
      start: start,
      end: end,
    });

    console.log(events);

    cursor = nextCursor;
    running = events.length > 0;
  }
}
```

**JavaScript**

```js
import {Configuration, ExportApi} from '@croct/export';

async function exportEvents() {
  const api = new ExportApi(
    new Configuration({
      apiKey: '<API KEY>'
    })
  );

  const today = new Date();
  today.setHours(0, 0, 0, 0);

  const yesterday = new Date(today.getTime());
  yesterday.setDate(today.getDate() - 1);

  const start = Math.floor(yesterday.getTime());
  const end = Math.floor(today.getTime());

  let cursor = undefined;
  let running = true;

  while (running) {
    const {data: {items: events, nextCursor}} = await api.exportEvents({
      pageSize: 100,
      cursor: cursor,
      start: start,
      end: end,
    });

    console.log(events);

    cursor = nextCursor;
    running = events.length > 0;
  }
}
```

**Java**

```java
import com.croct.client.export.ApiClient;
import com.croct.client.export.ApiException;
import com.croct.client.export.api.ExportApi;
import com.croct.client.export.model.Event;
import com.croct.client.export.model.EventResponse;
import java.time.LocalDate;
import java.time.ZoneId;
import java.util.List;

public class StatelessPagination {
  public static void main(String[] args) throws ApiException {
    final ApiClient client = new ApiClient();
    client.setApiKey("<API KEY>");

    final ExportApi api = new ExportApi(client);

    final LocalDate today = LocalDate.now();
    final LocalDate yesterday = today.minusDays(1);

    final long start = yesterday
      .atStartOfDay(ZoneId.systemDefault())
      .toInstant()
      .toEpochMilli();

    final long end = today
      .atStartOfDay(ZoneId.systemDefault())
      .toInstant()
      .toEpochMilli();

    String cursor = null;
    final int pageSize = 100;

    while (true) {
      final EventResponse response = api.exportEvents(
        start,
        end,
        pageSize,
        cursor,
        null
      );

      final List<Event> events = response.getItems();

      System.out.println(events);

      if (events.isEmpty()) {
        break;
      }

      cursor = response.getNextCursor();
    }
  }
}
```