Skip to main contentOverview
Incremental syncing involves transferring only new or updated data, thus avoiding duplication of previously replicated data. This is achieved through deduplication using a unique primary key specified in the model. For initial syncs, it functions like a full refresh since all data is considered new.
Example
Initial State
Suppose the following records already exist in our source:
| Name | Plan | Updated At |
|---|
| Charles Beaumont | freemium | 6789 |
| Eleanor Villiers | freemium | 6790 |
First sync
In this sync, the delta contains an updated record for Charles:
| Name | Plan | Updated At |
|---|
| Charles Beaumont | freemium | 6791 |
After this incremental sync, the data in the warehouse would now be:
| Name | Plan | Updated At |
|---|
| Charles Beaumont | freemium | 6791 |
| Eleanor Villiers | freemium | 6790 |
Second sync
Let’s assume in the next delta both customers have upgraded to a paid plan:
| Name | Plan | Updated At |
|---|
| Charles Beaumont | paid | 6795 |
| Eleanor Villiers | paid | 6795 |
The final data at the destination after this update will be:
| Name | Plan | Updated At |
|---|
| Charles Beaumont | paid | 6795 |
| Eleanor Villiers | paid | 6795 |
