Real-time syncing with webhooks
Guide to sync data in real-time with webhooks.
Pre-requisites:
- Configure webhooks in the external API portal (details).
- Set up a custom sync integration
What is a real-time sync
Real-time sync involves receiving webhooks from external APIs and processing them in Nango via syncs. By default, syncs poll data at a set frequency, but they can also be configured to handle webhooks for real-time updates.
When processed in a sync:
- The webhook triggers the execution of a Nango sync script
- The script extracts data from the webhook and optionally fetches additional data from the external API
- The modified data is stored in the Nango cache
henever the cache is updated, Nango sends a sync completion webhook to notify your application of new data. This entire process happens in real-time.
Webhooks and periodic polling:
Webhooks can fail due to external API outages or unordered events. Periodic polling ensures data consistency by reconciling any missed updates.
Implement a real-time sync
To enable real-time sync, define webhook-subscriptions
in your nango.yaml
under the relevant sync
configuration:
Use '*'
to list for any webhook subscription, e.g.
Handle payloads in your sync script, e.g.
After processing an external webhook, Nango sends a POST
request to your app with the following body:
To check if a sync supports webhooks, navigate to the Integrations tab > select an integration > Endpoints sub-tab > check the script settings for webhook subscriptions.
Sync concurrency considerations
When multiple syncs or webhooks modify the same records simultaneously, data conflicts can arise. Nango provides merging strategies to manage concurrency.
Key Concepts
- Race conditions: Webhooks and syncs may attempt concurrent modifications, causing inconsistencies.
- Modification tracking: Nango records when a record was last modified to prevent unintended overwrites.
- Configurable behavior: Define whether newer updates should be preserved or overwritten.
Available Strategies
Use setMergingStrategy
to control sync behavior:
- override (default): Updates records regardless of modification time.
- ignore_if_modified_after: Prevents older batch operations from overwriting newer records.
Example usage
Strategy details
override
(default)
- Applies updates regardless of last modification time.
- Useful when sync script data should take priority.
- Warning: May overwrite real-time updates from webhooks.
ignore_if_modified_after
- Preserves records modified after the last batch operation.
- Recommended for real-time webhook updates.
Best Practices
- Batch records sequentially:
batchSave
,batchUpdate
, andbatchDelete
operations should run sequentially. - Persist data promptly: Fetched data should be saved in the next
batchSave
,batchUpdate
, orbatchDelete
call. - Avoid concurrent fetch operations: Do not fetch data concurrently with
setMergingStrategy
or batch operations.
Questions, problems, feedback? Please reach out in the Slack community.
Was this page helpful?