Pre-requisites:

What is a real-time sync

Real-time sync involves receiving webhooks from external APIs and processing them in Nango via syncs. By default, syncs poll data at a set frequency, but they can also be configured to handle webhooks for real-time updates.

When processed in a sync:

  • The webhook triggers the execution of a Nango sync script
  • The script extracts data from the webhook and optionally fetches additional data from the external API
  • The modified data is stored in the Nango cache

henever the cache is updated, Nango sends a sync completion webhook to notify your application of new data. This entire process happens in real-time.

Webhooks and periodic polling:

Webhooks can fail due to external API outages or unordered events. Periodic polling ensures data consistency by reconciling any missed updates.

Implement a real-time sync

To enable real-time sync, define webhook-subscriptions in your nango.yaml under the relevant sync configuration:

hubspot:
  syncs:
    hubspot-contacts-sync:
	    webhook-subscriptions:
			  - contact.propertyChange

Use '*' to list for any webhook subscription, e.g.

airtable:
  syncs:
    airtable-sync-name:
	    webhook-subscriptions:
			  - '*'

Handle payloads in your sync script, e.g.

// Use for periodic polling.
export default async function fetchData(nango: NangoSync) {
  // Use for periodic polling.
}

// Use for webhook processing.
export async function onWebhookPayloadReceived(nango: NangoSync, payload: any): Promise<void> {
    if (payload.subscriptionType === 'contact.propertyChange') {
        const updatedObject = {
            id: payload.objectId,
            [payload.propertyName]: payload.propertyValue
        };

		// Use nango.batchSave() or nango.batchUpdate() to save/update records.
    }
}

After processing an external webhook, Nango sends a POST request to your app with the following body:

{
    "connectionId": "<CONNECTION-ID>",
    "providerConfigKey": "<INTEGRATION-ID>",
    "syncName": "<SYNC-NAME>",
    "model": "<MODEL-NAME>",
    "responseResults": { "added": 1, "updated": 0, "deleted": 0 },
    "syncType": "WEBHOOK",
    "modifiedAfter": "<TIMESTAMP>"
}

To check if a sync supports webhooks, navigate to the Integrations tab > select an integration > Endpoints sub-tab > check the script settings for webhook subscriptions.

Sync concurrency considerations

When multiple syncs or webhooks modify the same records simultaneously, data conflicts can arise. Nango provides merging strategies to manage concurrency.

Key Concepts

  • Race conditions: Webhooks and syncs may attempt concurrent modifications, causing inconsistencies.
  • Modification tracking: Nango records when a record was last modified to prevent unintended overwrites.
  • Configurable behavior: Define whether newer updates should be preserved or overwritten.

Available Strategies

Use setMergingStrategy to control sync behavior:

  1. override (default): Updates records regardless of modification time.
  2. ignore_if_modified_after: Prevents older batch operations from overwriting newer records.

Example usage

export default async function fetchData(nango: NangoSync) {
    // Set merging strategy at the start of the script
    await nango.setMergingStrategy(
        { strategy: 'ignore_if_modified_after' },
        'Contact'
    );

    // Persist records to the Nango cache.
    const contacts: Contact[] = ...
    await nango.batchSave(contacts, 'Contact');
}

Strategy details

override (default)

  • Applies updates regardless of last modification time.
  • Useful when sync script data should take priority.
  • Warning: May overwrite real-time updates from webhooks.
// This is the default behavior if no merging strategy is set in the script
await nango.setMergingStrategy(
    { strategy: 'override' },
    'Contact'
);

ignore_if_modified_after

  • Preserves records modified after the last batch operation.
  • Recommended for real-time webhook updates.
await nango.setMergingStrategy(
    { strategy: 'ignore_if_modified_after' },
    'Contact'
);

Best Practices

  1. Batch records sequentially: batchSave, batchUpdate, and batchDelete operations should run sequentially.
  2. Persist data promptly: Fetched data should be saved in the next batchSave, batchUpdate, or batchDelete call.
  3. Avoid concurrent fetch operations: Do not fetch data concurrently with setMergingStrategy or batch operations.

Questions, problems, feedback? Please reach out in the Slack community.