Skip to main content

Generate Test Data

NineData can generate simulated data that matches selected business rules and table relationships. Use generated data for functional testing, stress testing, migration validation, and other non-production verification work without using real customer data.

Feature Description​

  • Custom Data Generation: Supports field generation rules and algorithms for specific business needs, helping the generated data follow real business logic. For example, the name field can generate names in a specific format, and the date field can generate dates within a specific range.

  • Data Volume Control: The quantity of generated data can be controlled, supporting the generation of data from a small scale to tens of millions of data records, meeting different testing needs.

  • Field Association and Dependency: Data can be automatically generated based on the association relationships between fields, ensuring the consistency of data between tables and fields, such as the maintenance of foreign key relationships.

  • Data Distribution Settings: Users can set the distribution ratio of generated data, including NULL value ratio, unique values, duplicate values, etc., helping to simulate the data distribution characteristics in the production environment.

Usage Scenarios​

  • Functional Testing: During the development process, test data is generated to verify whether the system's functions operate as expected. For example, in user registration functional testing, a batch of simulated user data can be generated to test the correctness of user registration, login, and data storage.

  • Performance Testing: By generating a large amount of data, the system's operation under high load is simulated, testing the system's response speed, resource consumption, and scalability. For example, generating millions of order data for performance testing to check the database's query and insertion performance.

  • Migration Functional Testing: When performing data migration or version upgrades, generating test data can verify whether the data migration process is correct, ensuring that the data remains complete and consistent in the environment after migration or upgrade.

Prerequisites​

  • The data source for test data generation has been added to NineData. For details, see Add Data Source.
  • The supported data source type is MySQL, Oracle, OceanBase for Oracle, OceanBase for MySQL, TiDB, GreatSQL, ClickHouse, Doris, StarRocks, TDSQL MySQL.

Create Data Generation Task​

  1. Log in to the NineData Console.

  2. Click on DevOps > Data Generation in the left navigation bar.

  1. Under the Data Generation tab, click Create Task in the toolbar, configure the parameters in the table, and click Create Task.

    ParameterDescription
    NameEnter the name of the data generation task.
    Datasource/DatabaseSelect the target data source and database (Schema) for the data generation task.
    TablesClick Add Tables and select the target table for the data generation task. Next to Tables, click the simulation template name to change the simulation template. Simulation templates support custom configuration. For details, see Manage Simulation Templates.
    After the table is added, the page automatically lists the table and all its fields.
    • Generated data volume: In the Generated data volume text box in the left table name area, you can configure the amount of data to be generated. This data volume is affected by the Data generation record number threshold configuration rule, with a maximum of 10 million records. For rule configuration methods, see Edit Policy.
    • Simulation Rule: In the right field name area, you can manually configure simulation rules for each field, and click Actions under the Configure Algorithm column of each field to configure simulation algorithms for each column.
    Conflict strategySelect the processing strategy for duplicate objects encountered during data generation, supporting Encounter conflict termination, Encounter conflict skip, Encounter conflict repalce three strategies.
    ExecutorNot displayed in personal mode. Select the person responsible for executing the current data generation task after approval.
    Description: The options in the Executor drop-down list depend on the following two situations:
    • The current data source has configured development specifications: The options in the list are based on the Generate task executor configuration rule configured in the current specification. This rule is located in the Data Generation tab. For information about configuring development specifications, see Edit Policy.
    • The current data source has not configured development specifications: The options in the list are users who have Data Generation Task - Submit permission for the current data source. For more information about authorization, see Configure User Permissions.
    Reason (Optional)Not displayed in personal mode. Records the purpose and background of the data generation task. In team collaboration, task auditing, and approval workflows, the reason helps reviewers understand why the task is needed.

  2. Enter the Pre-check node, the system will pre-audit the task based on the SQL Development Specifications associated with the current data source, and there will be the following results:

    • Pre-audit passed: Depending on the configuration of the approval process, the task status will change to Pending or Approved. If it is the former, continue to the next step. If it is the latter, this process ends.
    • Pre-audit not passed: The task status will change to Pre-Check Failed, you can Withdraw Data Generation Task to re-edit and submit.

Withdraw Data Generation Task​

When the task status is Pre-Checking, Pending Approval, Approved, or Scheduled, you can withdraw the application for the task. After withdrawal, the task will no longer be executed, and you can re-edit the task form and submit it again.

Prerequisites

  • The task status is Pre-Checking, Pending Approval, Approved, or Scheduled.
  • You are the Submitter or Administrator of the target task.

Operation Steps

  1. Log in to the NineData Console.

  2. Click on DevOps > Data Generation in the left navigation bar.

  1. Under the Data Generation tab, find the target task, click the task ID or the Actions column of the task, and click Details. Click Withdraw in the toolbar of the Details page.

  2. In the pop-up confirmation window, click Withdraw or Withdraw and Edit.

    • Withdraw: The task status changes to Withdraw. If you need to re-edit the task, find it in the Data Generation list, click Duplicate under the Actions column, and submit the task form again.
    • Withdraw and Edit: The task status changes to Withdraw and automatically jumps to the task editing page.

Approve Data Generation Task​

Prerequisites

  • The task status is Pending Approval.
  • You are the Approver or Administrator of the target task.

Operation Steps

  1. Log in to the NineData Console.

  2. Click on DevOps > Data Generation in the left navigation bar.

  1. Under the Data Generation tab, find the target task, click the task ID or the Actions column of the task, and click Details.

  2. On the Details page, approve the task, and choose Transfer, Approved, or Rejected based on your decision.

    Operation    		Description
    TransferTransfer the task to another user for approval.
    ApprovedApprove the application, click Approved, you can enter Reason as needed, and click OK.
    RejectedReject the application and do not execute the task. Click Rejected, you also need to enter Reason for rejection, and click OK.

Execute Data Generation Task​

When the task status is Approved, you can execute the task.

Prerequisites

  • The task status is Approved.
  • You are the Executor or Administrator of the target task.

Operation Steps

  1. Log in to the NineData Console.

  2. Click on DevOps > Data Generation in the left navigation bar.

  1. Under the Data Generation tab, click the task ID whose status is Approved.

  2. On the Details page, click Execute in the toolbar.

  3. In the pop-up window, select Execution Method, click OK, and then wait for the task status to change to Success.

    • Execute Now: Execute the SQL statement immediately.
    • Scheduled Execution: Choose a time to execute the SQL statement.

    After successful execution, you can open the SQL Console to verify the generated data.

Result​

The data generation task creates simulated records in the selected target tables. Verify the generated data in SQL Console before using it for functional tests, performance tests, or migration validation.