Skip to main content

Use advanced data archiving options

NineData Data Archiving can archive expired data, clean source tables, and reclaim table space through scheduled tasks. Advanced options help teams make archiving safer for global databases, varied time fields, and long-running cleanup jobs.

Archiving Workflow​

A typical archive and cleanup workflow is:

  1. Identify data that is ready to be archived in the business database.
  2. Move the data to an archive database.
  3. Clean the archived data from the business database.
  4. Optimize the business table to reclaim space when required.

Data archiving workflow

Advanced Options​

NineData provides the following options for data archiving and cleanup tasks:

  • Time zone selection: Select the time zone of the target data source before configuring task start time.
  • Multiple time field types: Use DATETIME, INT, BIGINT, CHAR, or VARCHAR fields as the archive time field.
  • Preview SQL: Review the archive and cleanup SQL generated from the current configuration before execution.
  • System variable: Use the bizdate variable to generate dynamic table names or filter conditions based on the current system time.
  • Optimize strategy: Configure optimize operations after cleanup to help reclaim table space.
  • Data conflict handling: Choose Ignore or Override when matching data already exists in the archive target.
  • Execution time limit: Set the maximum execution window for periodic tasks so cleanup work does not continue into business peak hours.

Before you begin​

Before you configure advanced archiving options, make sure that:

  • The source and archive target data sources have been added to NineData.
  • A data archiving task has been planned or created.
  • You know the archive time field, retention policy, and target table naming rule.
  • You have permission to create or edit data archiving tasks.
  • Any required approval workflow is configured for the target environment.

Configure Task-Level Options​

Use the first task configuration page to set options such as time zone, optimize strategy, execution time, and data conflict handling.

Configure advanced archive task options

  • Time Zone: Subsequent task execution is based on the selected time zone.
  • Optimize Strategy: Configure how optimize operations are performed after cleanup.
  • Execution Time: Define the maximum allowed execution duration. If the task exceeds the limit, NineData stops the task to reduce business impact.
  • Data Conflict Handling Strategy: Choose how NineData handles rows that already exist in the archive target.

Configure Variables, Time Fields, And Preview SQL​

Use mapping and filtering settings to configure system variables, dynamic table names, supported time fields, and SQL preview.

Configure system variables and preview SQL

  • System Variable: Configure the format of bizdate. The default format is yyyyMMdd, such as 20240617. Other formats, such as yyyy-MM-dd, are also supported.
  • Dynamic table name: Use a variable in the target table name. For example, customers$(bizdate) creates a table name such as customers20240617.
  • Time Field: Select a supported time field type, including INT, BIGINT, CHAR, and VARCHAR.
  • Preview SQL: Review the archive and cleanup SQL before the task runs.

Result​

Advanced options make archiving tasks easier to align with business windows, global time zones, field design differences, and archive conflict rules. Review the task details after execution to confirm archived rows, cleanup status, optimize behavior, and errors.

Next Steps​