What is a real-time sync
Real-time sync involves receiving webhooks from external APIs and processing them in Nango via syncs. By default, syncs poll data at a set frequency, but they can also be configured to handle webhooks for real-time updates. When processed in a sync:- The webhook triggers the execution of a Nango sync script
- The script extracts data from the webhook and optionally fetches additional data from the external API
- The modified data is stored in the Nango cache
Webhooks & periodic polling
Webhooks can fail due to external API outages or unordered events. Periodic polling ensures data consistency by reconciling any missed updates. If you use webhooks for realtime syncs, we recommend to always combine them with a polling sync.Near realtime syncing
As an alternative to real-time syncing, consider near-real-time syncing by polling at a higher frequency, which is simpler. Our high-frequency syncing add-on allows polling every 30 seconds. This is more resource intensive, but can be a good compromise if you are only syncing data for a handful of Connections.Implement a realtime sync
Step 1 - Setup prerequisites
To create a realtime sync, you need to have:- Webhooks from the external API enabled. Follow our implement webhooks processing guide.
- A sync for the object you want to sync. Follow our implement a sync guide.
Step 2 - Enable webhooks processing
To enable the real-time sync, definewebhookSubscriptions
and onWebhook
in your sync function:
"syncType": "WEBHOOK"
.
To check if a sync supports webhooks, navigate to the Integrations tab > select an integration > Endpoints sub-tab > check the script settings for webhook subscriptions.
Sync concurrency considerations
When multiple syncs or webhooks modify the same records simultaneously, data conflicts can arise. Nango provides merging strategies to manage concurrency.Key Concepts
- Race conditions: Webhooks and syncs may attempt concurrent modifications, causing inconsistencies.
- Modification tracking: Nango records when a record was last modified to prevent unintended overwrites.
- Configurable behavior: Define whether newer updates should be preserved or overwritten.
Available Strategies
UsesetMergingStrategy
to control sync behavior:
- override (default): Updates records regardless of modification time.
- ignore_if_modified_after: Prevents older batch operations from overwriting newer records.
Example usage
- TypeScript
Strategy details
override
(default)
- Applies updates regardless of last modification time.
- Useful when sync script data should take priority.
- Warning: May overwrite real-time updates from webhooks.
ignore_if_modified_after
- Preserves records modified after the last batch operation.
- Recommended for real-time webhook updates.
Best Practices
- Batch records sequentially:
batchSave
,batchUpdate
, andbatchDelete
operations should run sequentially. - Persist data promptly: Fetched data should be saved in the next
batchSave
,batchUpdate
, orbatchDelete
call. - Avoid concurrent fetch operations: Do not fetch data concurrently with
setMergingStrategy
or batch operations.
Questions, problems, feedback? Please reach out in the Slack community.