Overview

Incremental syncing involves transferring only new or updated data, thus avoiding duplication of previously replicated data. This is achieved through deduplication using a unique primary key specified in the model. For initial syncs, it functions like a full refresh since all data is considered new.

Example

Initial State

Suppose the following records already exist in our source:

NamePlanUpdated At
Charles Beaumontfreemium6789
Eleanor Villiersfreemium6790

First sync

In this sync, the delta contains an updated record for Charles:

NamePlanUpdated At
Charles Beaumontfreemium6791

After this incremental sync, the data in the warehouse would now be:

NamePlanUpdated At
Charles Beaumontfreemium6791
Eleanor Villiersfreemium6790

Second sync

Let’s assume in the next delta both customers have upgraded to a paid plan:

NamePlanUpdated At
Charles Beaumontpaid6795
Eleanor Villierspaid6795

The final data at the destination after this update will be:

NamePlanUpdated At
Charles Beaumontpaid6795
Eleanor Villierspaid6795