Pagination and incremental sync

1. The first full walk

Walk every page of GET /passports and store each record by its id. Note the time you started — not finished — as the high-water mark.

// Node.js 18+. An async iterator over every passport, page by page.const BASE = 'https://dppapi-production.up.railway.app/v1';async function* passports(filters = {}) {let cursor;do {const qs = new URLSearchParams({ ...filters, limit: '200', ...(cursor && { cursor }) });const res = await fetch(`${BASE}/passports?${qs}`, {headers: { Authorization: `Bearer ${process.env.DPP_API_KEY}` },});if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);const page = await res.json();yield* page.data;cursor = page.has_more ? page.next_cursor : undefined;} while (cursor);}for await (const passport of passports({ model_id: 'cm1q8x2k40003l70f9a8s7d6e' })) {console.log(passport.id, passport.status);}

2. Deltas with updated_since

On every later run, walk only what changed since the high-water mark. updated_since on GET /passports also returns passports whose only change is a new dynamic-data event, so one query covers status, publication and telemetry.

curl "https://dppapi-production.up.railway.app/v1/passports?updated_since=2027-06-01T00%3A00%3A00Z&limit=200" \-H "Authorization: Bearer $DPP_API_KEY"

• Upsert by id — a record can appear in two consecutive deltas.
• Overlap the windows by a few minutes (use mark − 5 min) so clock skew and in-flight writes are never missed.
• Advance the mark only after the whole walk succeeded.
• For new events, follow up with GET /passports/{id}/events?since= for each changed passport.
• Models change rarely: sync GET /models?updated_since= the same way.

3. Webhooks for immediacy

Subscribe to passport.* and model.* webhooks and refresh the record named in each event. Keep the delta loop running hourly or nightly anyway: it repairs anything a webhook outage missed, and it is cheap when nothing changed.

Budget

At 200 passports a page and 600 requests a minute, a full walk reads 120,000 passports a minute. If you need a complete snapshot rather than a mirror, the export archive is one request.