Generate Test Data

NineData supports the automatic generation of random data in the database that conforms to specific business scenarios, simulating the actual production environment data situation, and helping users perform functional testing, stress testing, and other verification work without using real data.

Feature Description

• Custom Data Generation: Supports customizing field generation rules and algorithms according to specific business needs, ensuring that the generated data complies with actual business logic. For example, the name field can generate names in a specific format, and the date field can generate dates within a specific range.

• Data Volume Control: The quantity of generated data can be controlled, supporting the generation of data from a small scale to tens of millions of data records, meeting different testing needs.

• Field Association and Dependency: Data can be automatically generated based on the association relationships between fields, ensuring the consistency of data between tables and fields, such as the maintenance of foreign key relationships.

• Data Distribution Settings: Users can set the distribution ratio of generated data, including NULL value ratio, unique values, duplicate values, etc., helping to simulate the data distribution characteristics in the production environment.

Usage Scenarios

• Functional Testing: During the development process, test data is generated to verify whether the system's functions operate as expected. For example, in user registration functional testing, a batch of simulated user data can be generated to test the correctness of user registration, login, and data storage.

• Performance Testing: By generating a large amount of data, the system's operation under high load is simulated, testing the system's response speed, resource consumption, and scalability. For example, generating millions of order data for performance testing to check the database's query and insertion performance.

• Migration Functional Testing: When performing data migration or version upgrades, generating test data can verify whether the data migration process is correct, ensuring that the data remains complete and consistent in the environment after migration or upgrade.

Prerequisites

• The data source that needs to generate test data has been added to NineData. For how to add it, please refer to Add Data Source.
• The supported data source type is MySQL, Oracle, OceanBase for Oracle, OceanBase for MySQL, TiDB, GreatSQL, ClickHouse, Doris, StarRocks, TDSQL MySQL.

Create Data Generation Task

1. Log in to the NineData Console.

2. Click on DevOps > Data Generation in the left navigation bar.

3. Under the Data Generation tab, click on Create Task in the upper right corner, and then configure the following parameters according to the table below, and click Create Task.

   Parameter	Description
   Name	Enter the name of the data generation task.
   Datasource/Database	Select the target data source and database (Schema) for the data generation task.
   Tables	Click Add Tables, select the target table for the data generation task. On the right side of Tables, you can click the name of the simulation template to change the simulation template. Simulation templates support custom configuration, please refer to Manage Simulation Templates. After the table is added, the page will automatically list the table and all its fields. Generated data volume: In the Generated data volume text box in the left table name area, you can configure the amount of data to be generated. This data volume is affected by the Data generation record number threshold configuration rule, with a maximum of 10 million records. For rule configuration methods, please refer to Edit Policy. Simulation Rule: In the right field name area, you can manually configure simulation rules for each field, and click Actions under the Configure Algorithm column on the right side of each field to configure simulation algorithms for each column.
   Conflict strategy	Select the processing strategy for duplicate objects encountered during data generation, supporting Encounter conflict termination, Encounter conflict skip, Encounter conflict repalce three strategies.
   Executor	【Not displayed in personal mode】The person responsible for executing the current data generation task after approval. Description: The options in the Executor drop-down list depend on the following two situations: The current data source has configured development specifications: The options in the list are based on the Generate task executor configuration rule configured in the current specification. This rule is located in the Data Generation tab, for how to configure development specifications, please refer to Edit Policy. The current data source has not configured development specifications: The options in the list are users who have Data Generation Task - Submit permission for the current data source. More about authorization, please refer to Configure User Permissions.
   Reason（Optional）	【Not displayed in personal mode】Used to record and explain the purpose and background of creating the data generation task, especially in team collaboration, task auditing, and approval processes, recording the reason for generation helps to verify the rationality of the task and promote the progress of the task flow.

4. Enter the Pre-check node, the system will pre-audit the task based on the SQL Development Specifications associated with the current data source, and there will be the following results:

   • Pre-audit passed: Depending on the configuration of the approval process, the task status will change to Pending or Approved. If it is the former, please continue to the next step. If it is the latter, this process ends.
   • Pre-audit not passed: The task status will change to Pre-Check Failed, you can Withdraw Data Generation Task to re-edit and submit.

Withdraw Data Generation Task

When the task status is Pre-Checking, Pending Approval, Approved, or Scheduled, you can withdraw the application for the task. After withdrawal, the task will no longer be executed, and you can re-edit the task form and submit it again.

Prerequisites

• The task status is Pre-Checking, Pending Approval, Approved, or Scheduled.
• You are the Submitter or Administrator of the target task.

Operation Steps

1. Log in to the NineData Console.

2. Click on DevOps > Data Generation in the left navigation bar.

3. Under the Data Generation tab, find the target task, click on the task ID or the Actions column on the right side of the task, and click Details. Click Withdraw in the upper right corner of the Details page.

4. In the pop-up confirmation window, click Withdraw or Withdraw and Edit.

   • Withdraw: The task status changes to Withdraw. If you need to re-edit the task, you can find the task in the Data Generation list, click on Duplicate under the Actions column on the right, and re-submit the task form.
   • Withdraw and Edit: The task status changes to Withdraw and automatically jumps to the task editing page.

Approve Data Generation Task

Prerequisites

• The task status is Pending Approval.
• You are the Approver or Administrator of the target task.

Operation Steps

1. Log in to the NineData Console.

2. Click on DevOps > Data Generation in the left navigation bar.

3. Under the Data Generation tab, find the target task, click on the task ID or the Actions column on the right side of the task, and click Details.

4. On the Details page, approve the task, and choose Transfer, Approved, or Rejected according to the actual situation.

   Operation	Description
   Transfer	Transfer the task to another user for approval.
   Approved	Approve the application, click Approved, you can input Reason as needed, and click OK.
   Rejected	Reject the application and do not execute the task. Click Rejected, you also need to input Reason for rejection, and click OK.

Execute Data Generation Task

When the task status is Approved, you can execute the task.

Prerequisites

• The task status is Approved.
• You are the Executor or Administrator of the target task.

Operation Steps

1. Log in to the NineData Console.

2. Click on DevOps > Data Generation in the left navigation bar.

3. Under the Data Generation tab, click on the task ID of the target status Approved.

4. On the Details page, click Execute in the upper right corner.

5. In the pop-up window, select Execution Method, click OK, and then wait for the task status to change to Success.

   • Execute Now: Execute the SQL statement immediately.
   • Scheduled Execution: Choose a time to execute the SQL statement.

   After successful execution, you can open the SQL Console to verify the generated data.