This guide has two parts:
  • How to create your own sync
  • How to use a sync in your own app
If you are using a pre-built reference implementation sync, you can skip to the second section.

How to build a sync

Step 1 - Initial Functions setup

If you don’t have a nango-integrations folder yet, follow the initial Functions setup guide first. Otherwise, you can skip to the next step.

Step 2 - Start dev mode

Before you plan to modify your integration functions, run:
nango dev # Keep the tab open
This command starts a process that continuously compiles your integration functions and prints code syntax errors.

Step 3 - Create the sync file

In your nango-integrations folder, create the file for your new sync function. Sync files should be within a syncs folder, which is nested under the integration’s folder. For example, if you want to create a new sync to sync contacts from salesforce, your structure should look like this:
nango-integrations/
├── .nango
├── .env
├── index.ts
├── package.json
└── salesforce # this is the integration id and must match an integration id in your Nango dashboard
    └── syncs/
        └── salesforce-contacts.ts # this is the name of your sync
In your sync file, paste the following scaffold:
salesforce-contacts.ts
import { createSync } from 'nango';
import * as z from 'zod';

const MyObject = z.object({
  id: z.string(),
  first_name: z.string(),
  last_name: z.string(),
  email: z.string(),
});

export default createSync({
    description: `<Description of your sync>`,
    version: '1.0.0', // Version, increment it when you release a new version
    endpoints: [{ method: 'GET', path: '/<integration>/<object-name>', group: '<Group>' }],
    frequency: 'every hour', // Default sync interval
    autoStart: true, // Should the sync start immediately when a new connection is created?
    syncType: 'full', // incremental or full (full refresh or incremental sync)
    trackDeletes: true, // detect deletes? See separate implementation guide
    models: {
        MyObject: MyObject,
    },
    exec: async (nango) => {
        // Integration code goes here.
    },
});
Also import your new sync file in your index.ts file:
index.ts
import './salesforce/syncs/salesforce-contacts';

Step 4 - Implement your sync

In the exec method, implement the logic of your sync. Edit MyObject to contain the properties you need. The following can help you with your implementation: Example implementation of the Salesforce contacts sync:
salesforce-contacts.ts
import { createSync } from 'nango';
import * as z from 'zod';

const SalesforceContact = z.object({
  id: z.string(),
  first_name: z.string(),
  last_name: z.string(),
  email: z.string(),
});

export default createSync({
  description: `Fetches contacts from Salesforce`,
  version: '1.0.0',
  endpoints: [{ method: 'GET', path: '/salesforce/contacts', group: 'Contacts' }],
  frequency: 'every hour',
  autoStart: true,
  syncType: 'full',
  models: {
    SalesforceContact: SalesforceContact,
  },
  exec: async (nango) => {
    const query = buildQuery(nango.lastSyncDate);
    await fetchAndSaveRecords(nango, query);
    await nango.log('Sync run completed!');
  },
});
export type NangoSyncLocal = Parameters<(typeof sync)['exec']>[0];

function buildQuery(lastSyncDate?: Date): string {
    let baseQuery = `SELECT Id, FirstName, LastName, Email, AccountId, LastModifiedDate FROM Contact`;

    if (lastSyncDate) { // Only fetch the new data.
        baseQuery += ` WHERE LastModifiedDate > ${lastSyncDate.toISOString()}`;
    }

    return baseQuery;
}

async function fetchAndSaveRecords(nango: NangoSyncLocal, query: string) {
    let endpoint = '/services/data/v53.0/query';

    while (true) {
        const response = await nango.get({
            endpoint: endpoint,
            params: endpoint === '/services/data/v53.0/query' ? { q: query } : {}
        });

        const mappedRecords = mapContacts(response.data.records);

        await nango.batchSave(mappedRecords, 'SalesforceContact'); // Saves records to Nango cache.

        if (response.data.done) {
            break;
        }

        endpoint = response.data.nextRecordsUrl;
    }
    await nango.deleteRecordsFromPreviousExecutions('SalesforceContact')
}

function mapContacts(records: any[]): SalesforceContact[] {
    return records.map((record: any) => {
        return {
            id: record.Id as string,
            first_name: record.FirstName,
            last_name: record.LastName,
            email: record.Email,
            account_id: record.AccountId,
            last_modified_date: record.LastModifiedDate
        };
    });
}
In this integration function, the following Nango utilities are used:
  • nango.lastSyncDate is the last date at which the sync has run
  • await nango.batchSave() to persist external data in Nango’s cache
  • await nango.get() to perform an API request (automatically authenticated by Nango)
  • await nango.log() to write custom log messages

Step 5 - Test your sync locally

Easily test your integration functions locally as you develop them with the dryrun function of the CLI:
nango dryrun salesforce-contacts '<CONNECTION-ID>'
To learn more about all the options for dryrun, run: nango dryrun --help Because this is a dry run, syncs won’t persist data in Nango. Instead, the retrieved data is printed to the console.
By default, dryrun retrieves connections from your dev environment. You can change this with a CLI flag.

Step 6 - Deploy your sync

To run your sync in Nango, you need to deploy it to an environment in your Nango account. To deploy all integrations in your nango-integrations folder, run:
nango deploy dev # dev is the name of the environment to which you are deploying
To only deploy a single sync, use the --sync parameter:
nango deploy --sync salesforce-contacts dev
Run nango deploy -h for more options to deploy only parts of your integrations. To fetch the synced data in your product, follow the steps in the next setion.
Most teams automate deployments to production in their CI.

How to use a sync

Pre-built reference implementations

For common use cases, pre-built reference implementations are available to help you get started fast. Select your integration in the Integrations tab, and navigate to the Endpoints tab. Available pre-built sync integrations will appear in the endpoint list. Select the relevant one and enable it with the toggle. Nango will automatically sync the corresponding data in the background for each relevant connection. Reference implementations are a starting point. You will likely need to customize them or create your own custom sync.

Step 1 - Setup webhooks from Nango

Nango sends webhook notifications to your backend whenever new data is available for a connection & sync combination. Set these up by following the implement webhooks from Nango guide. When the sync finishes, Nango will send you a webhooks with this payload.

Step 2 - Fetch the latest data from Nango

After receiving a Nango webhook, fetch the latest records using the backend SDK (reference) or API (reference). Use the modifiedAfter timestamp from the webhook payload as a parameter in your request to fetch only the modified records.
curl -G https://api.nango.dev/records \
--header 'Authorization: Bearer <ENVIRONMENT-SECRET-KEY>' \
--header 'Provider-Config-Key: <providerConfigKey-in-webhook-payload>' \
--header 'Connection-Id: <connectionId-in-webhook-payload>' \
--data-urlencode 'model=<model-in-webhook-payload>' \
--data-urlencode 'modified_after=<modifiedAfter-in-webhook-payload>' \
This returns an array of records conforming to the specified data model. Each record contains useful metadata automatically generated by Nango:
{
    records:
        [
            {
                id: 123,
                ..., // Fields as specified in the model you queried
                _nango_metadata: {
                    deleted_at: null,
                    last_action: 'ADDED',
                    first_seen_at: '2023-09-18T15:20:35.941305+00:00',
                    last_modified_at: '2023-09-18T15:20:35.941305+00:00',
                    cursor: 'MjAyNC0wMy0wNFQwNjo1OTo1MS40NzE0NDEtMDU6MDB8fDE1Y2NjODA1LTY0ZDUtNDk0MC1hN2UwLTQ1ZmM3MDQ5OTdhMQ=='
                }
            },
            ...
        ],
    next_cursor: "Y3JlYXRlZF9hdF4yMDIzLTExLTE3VDExOjQ3OjE0LjQ0NyswMjowMHxpZF4xYTE2MTYwMS0yMzk5LTQ4MzYtYWFiMi1mNjk1ZWI2YTZhYzI"
}

Cursor-based synchronization

In practice, webhook notifications can be missed, and relying solely on the webhook payload to fetch modified records can cause you to miss some updates. A more reliable way of keeping track of how far you’ve synced records (for each connection & sync combination) is to rely on record cursors. Each record comes with a synchronization cursor in _nango_metadata.cursor. Nango uses cursors internally to keep a chronological list of record modifications. Each time you fetch records, you should store the cursor of the last record you fetched to remember how far you’ve synced (for each connection & sync combination). The next time you fetch records, pass in the cursor of the last-fetched record to only receive records modified after that record:
curl -G https://api.nango.dev/records \
--header 'Authorization: Bearer <ENVIRONMENT-SECRET-KEY>' \
--header 'Provider-Config-Key: <providerConfigKey-in-webhook-payload>' \
--header 'Connection-Id: <connectionId-in-webhook-payload>' \
--data-urlencode 'model=<model-in-webhook-payload>' \
--data-urlencode 'cursor=<cursor-of-last-fetched-record>' \
So, the overall logic for cursor-based synchronization should be:
  1. Receive a webhook notification from Nango
  2. Query your database for the cursor of the last-fetched record
  3. Fetch the modified records (passing the cursor)
  4. Store the modified records
  5. Store the last-fetched record cursor