Guide on how to use sync variants to partition data and optimize sync execution.
Sync partitioning enables more flexible and scalable data synchronization by allowing you to create multiple independent “variants” of the same sync. This feature is particularly useful when dealing with large datasets, multiple tenants, or different filtering requirements within the same integration.
This feature is available on the Growth plan and on Enterprise plans. See Pricing.
Traditionally, a sync is defined once in a TypeScript file and executed for each connection. With sync partitioning, you can create multiple sync “variants” that share the same base configuration but run independently. Each variant has its own execution schedule, last sync timestamp, and data storage.Key benefits:
Parallel execution: Variants run independently, allowing more granular control over sync execution.
Custom filtering: Each variant can target different subsets of data.
Better resource utilization: Distribute sync load across multiple partitions instead of running a single, monolithic sync.
Tenant-based partitioning: Useful for multi-tenant applications where data is synced per customer.
By default, a sync is associated with a base variant.
You can create additional sync variants programmatically using the API or SDK.
Each variant is treated as a separate sync in the Nango UI.
Sync webhooks include a variant field to identify which variant was executed.
External webhooks (e.g., from Airtable) are always processed by the base variant.
If a sync variant needs additional context (e.g., filtering parameters), store/retrieve it using the connection metadata, namespaced with the variant ID.
Once created, a variant can be managed just like a regular sync. All sync operations (triggering, pausing, resuming, etc.) support the variant parameter.Check out the HTTP API reference or Node SDK reference for details.
When a sync script runs, the variant name is available via nango.variant. You can use this to customize the behavior of your sync.
Copy
Ask AI
export default createSync({ exec: async (nango) => { await nango.log(`Running sync with variant: ${nango.variant}`); // Use the variant name to namespace the metadata and/or customize API calls. const res = await nango.get({ endpoint: `/orders?filter=${nango.variant}` }); // Records are saved in the context of this specific variant. await nango.batchSave(res.data.orders, 'Order'); },});
If a sync variant requires custom parameters (e.g., a filtering threshold), store them in the connection metadata, namespaced by the variant ID.Storing metadata: