Incremental

Overview

Incremental syncing involves transferring only new or updated data, thus avoiding duplication of previously replicated data. This is achieved through deduplication using a unique primary key specified in the model. For initial syncs, it functions like a full refresh since all data is considered new.

Example

Initial State

Suppose the following records already exist in our source:

Name	Plan	Updated At
Charles Beaumont	freemium	6789
Eleanor Villiers	freemium	6790

First sync

In this sync, the delta contains an updated record for Charles:

Name	Plan	Updated At
Charles Beaumont	freemium	6791

After this incremental sync, the data in the warehouse would now be:

Name	Plan	Updated At
Charles Beaumont	freemium	6791
Eleanor Villiers	freemium	6790

Second sync

Let’s assume in the next delta both customers have upgraded to a paid plan:

Name	Plan	Updated At
Charles Beaumont	paid	6795
Eleanor Villiers	paid	6795

The final data at the destination after this update will be:

Name	Plan	Updated At
Charles Beaumont	paid	6795
Eleanor Villiers	paid	6795

Overview

Sources

Destinations

Data Modelling

Data Syncs

Overview

Example

Initial State

First sync

Second sync

​Overview

​Example

​Initial State

​First sync

​Second sync

Overview

Example

Initial State

First sync

Second sync