Syncs
Incremental
Incremental sync only transfer new or updated data, minimizing data transfer
Overview
Incremental syncing involves transferring only new or updated data, thus avoiding duplication of previously replicated data. This is achieved through deduplication using a unique primary key specified in the model. For initial syncs, it functions like a full refresh since all data is considered new.
Example
Initial State
Suppose the following records already exist in our source:
Name | Plan | Updated At |
---|---|---|
Charles Beaumont | freemium | 6789 |
Eleanor Villiers | freemium | 6790 |
First sync
In this sync, the delta contains an updated record for Charles:
Name | Plan | Updated At |
---|---|---|
Charles Beaumont | freemium | 6791 |
After this incremental sync, the data in the warehouse would now be:
Name | Plan | Updated At |
---|---|---|
Charles Beaumont | freemium | 6791 |
Eleanor Villiers | freemium | 6790 |
Second sync
Let’s assume in the next delta both customers have upgraded to a paid plan:
Name | Plan | Updated At |
---|---|---|
Charles Beaumont | paid | 6795 |
Eleanor Villiers | paid | 6795 |
The final data at the destination after this update will be:
Name | Plan | Updated At |
---|---|---|
Charles Beaumont | paid | 6795 |
Eleanor Villiers | paid | 6795 |
Was this page helpful?