Periodic Full Replication

NineData's data replication feature supports periodic full replication between two data sources, automatically replicating all data from the source end to the target end according to a predefined schedule.

Background Information

To effectively protect the core data of an enterprise, it is common for businesses to implement robust backup and synchronization solutions, ensuring that data from core business databases is replicated entirely to target systems. As business databases are continuously updated, businesses need to execute replication tasks on a regular cycle based on configured intervals, such as daily, hourly, or even every ten minutes, to keep the data in the target system up to date.

Prerequisites

• The data sources participating in periodic full replication have been added to NineData. For instructions on how to add data sources, please refer to Adding Data Sources.
• The target replication link supports full replication. For a list of supported replication links, please refer to Data Replication.

Procedure

This article uses MySQL to MySQL replication link as an example to illustrate the configuration method for periodic full replication.

1. Log in to the NineData Console.

2. In the left navigation pane, click > .

3. On the page, click .

4. On the tab, configure as per the table below, and click .

   Parameter	Description
   	Enter a name for the data synchronization task. For ease of subsequent retrieval and management, please use a meaningful name. Maximum 64 characters supported.
   	The data source where the synchronization object resides.
   	The data source receiving the synchronized object.
   	Select .
   	Check and , then click the slider switch on the right of to enable periodic full replication. Note: Since periodic full replication cannot run simultaneously with , do not check .
   	The execution frequency of periodic full replication, supporting , , and . Please select according to actual business needs.
   	Select the start time point of periodic full replication. You can choose the time zone according to the region of your business needs.
   	Choose the data processing strategy in the target data source before each full replication starts. (optional when is selected): Deletes (DROP TABLE) the target table objects in the target data source, then rebuilds and writes data based on the structure of the source table. (optional when is not selected): Clears (TRUNCATE TABLE) all data in the target table in the target data source, then writes based on the data of the source table. : Directly overwrites (REPLACE) the data of the target table based on the data of the source table.

5. On the tab, configure the following parameters, then click .

   Parameter	Description
   	Select the content to be copied. You can choose to copy all content of the source database, or choose , select the content to be copied in the list, and click > to add it to the right list.
   (optional)	Click to add a blacklist record, select the database or object to be added to the blacklist, and these contents will not be copied. Used to exclude certain databases or objects from full replication of or replication of . Left dropdown: Select the name of the database to be added to the blacklist. Right dropdown: Select the objects in the corresponding database. You can click on multiple objects for multiple selections, leave it blank to add the entire database to the blacklist. If you want to add multiple databases to the blacklist, you can click the Add button below to add a row.

6. On the tab, select different operations according to the selected replication type.

   • Including : Configure the table name after synchronization to the target data source, and click .

     tip

     You can click on the on the right side of the page to customize the column names after synchronization to the target data source. In addition, you can set to synchronize only the data that meets the filtering conditions to the target data source. Taking the test data table employees as an example, if you set the filter condition to emp_no>=10005, then data with emp_no less than 10005 will not be synchronized to the target data source.

   • Excluding : The system defaults to selecting the same database in the target data source. If it does not exist, you need to manually select the target database. The table names and column names in the target database need to be consistent with the synchronized objects. If they are inconsistent, you can manually map the table names and column names.

7. On the tab, wait for the system to complete the pre-check. After the pre-check passes, click .

   tip

   • You can check . After the synchronization task is completed, automatically start data consistency comparison based on the source data source to ensure data consistency on both ends. Depending on the selected , the timing of the startup of is as follows:
     • : Starts after the structure replication is completed.
     • + , : Starts after full replication is completed.
   • If the pre-check fails, click the on the right side of the target check item, troubleshoot the cause of the failure, manually fix it, and click to perform the pre-check again until it passes.
   • For check items with as , you can fix or ignore them according to the specific situation.

8. On the page, when prompted with , the synchronization task starts running. At this point, you can:

   • Click to view the execution status of each stage of the synchronization task.
   • Click to return to the task list page.