Skip to main content

Managing Sensitive Data

NineData supports setting one or more columns in a data source as sensitive columns. Users who are not authorized to view sensitive columns will not be able to see the content of these columns.

Feature Description​

NineData's sensitive data feature is part of database management, designed to ensure that specific sensitive information stored in the database is not accessed by unauthorized users. It supports both manual and automatic addition of sensitive columns.

The automatic addition of sensitive columns is composed of several key components: Sensitivity Level, Data Type, and Masking Algorithm.

  • Sensitivity Level: Provides six levels from S0 to S5; the higher the number, the higher the security level. S0 indicates non-sensitive fields. Each level from S1 to S5 corresponds to different approval processes, allowing administrators to configure various approval strategies for classified and hierarchical management of sensitive data.
  • Data Type: The type of sensitive data. The system provides 27 data types by default. Each data type is associated with Sensitivity Level, Masking Algorithm, and Detection Rules of sensitive fields. Data Type is used to automatically identify sensitive fields in tables. Once Detection Rules matches the target column, it will associate with that column, automatically classifying and grading the sensitive columns.
  • Masking Algorithm: The masking method for sensitive columns. Data in sensitive columns will be encrypted using this algorithm. The system provides 33 masking algorithms by default and supports creating custom masking algorithms based on actual business needs.

Based on the above three components, the system scans the data sources, supporting full-database scanning and specified-database scanning. It also supports single scans and periodic scans, facilitating the automated configuration of sensitive columns after new fields are added.

Additionally, the sensitive data feature provides the Dashboard function, displaying information related to sensitive data within the current organization. This includes the total number of data sources supporting sensitive data protection, the total number of data sources with sensitive data enabled and their sensitivity levels, the total number of tables with sensitive data enabled, the total number of sensitive columns, the number of sensitive data accesses, and more. For more information, see Viewing the Sensitive Data Dashboard.

Prerequisites​

  • You have created or joined an organization, and this organization has subscribed to either DevOps Pro or DevOps Enterprise. Please ensure that your annual or monthly subscription is still active. For more information, please refer to Manage Organizations.
  • Your current account has switched to the target organization. For more information, please refer to Switching to an Organization.

  • The data source requiring sensitive fields has been added to NineData. For instructions on how to add, see Adding Data Sources.

  • The supported data source types for sensitive data are MySQL, SQL Server, PostgreSQL, Oracle, OceanBase Oracle, Db2, TiDB, OceanBase MySQL, GreatSQL, MariaDB, ClickHouse, Doris, SelectDB, StarRocks, SingleStore, Lindorm, and AnalyticDB for PostgreSQL, as well as TDSQL for MySQL.

  • To set sensitive columns for views, the data source type must be MySQL, Db2, PostgreSQL, TiDB, or TDSQL MySQL version.

  • To use the automatic addition of sensitive columns feature, please ensure the data source type is MySQL (Including MySQL-like data sources), SQL Server, PostgreSQL, Oracle.

Precautions​

  • The Administrator role can view all sensitive columns without authorization.
  • Under DevOps Pro, only up to three data sources can be configured with sensitive columns; DevOps Enterprise has no such limitation.

Automatically Adding Sensitive Columns to Data Sources​

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.

  1. On the Datasource tab, the system automatically lists all data sources that support adding sensitive data. Find the target data source and click Scan Setting under the Actions column on its right side more.

    • If you want to quickly start scanning with default configurations, you can click Immediate Scan under the Actions column on the right side of the target data source more, then choose whether to take effect immediately after the scan is completed in Scan Results Activate Immediately.

    • To scan multiple data sources in batch, you can select the checkboxes on the left side of the target data sources, then click Immediate Scan or Scan Setting at the top of the page.

  2. Configure each parameter according to the table below, and click OK.

    ParameterDescription
    Scanning ObjectThe scanning scope of sensitive columns, supporting All Databases and Specified Databases.
    • All Databases: Scan all databases and tables in the current data source.
    • Specified Databases: Manually specify some databases. After selecting this option, you need to select databases below, supporting single and multiple selections.
    Scan ModeSupports Single Scan and Periodic Scan.
    • Single Scan: Perform a one-time scan of sensitive columns in the database.
    • Periodic Scan: Periodically scan sensitive columns in the database.
    Time ZoneThe time zone information for the start time (Excuted Time) of sensitive column scanning, used in conjunction with Excuted Time.
    • Single Scan scenario: Optional. If Excuted Time is not specified, there is no need to configure the time zone.
    • Periodic Scan scenario: Required. Select the time zone information for the scanning start time.
    Periodic (Visible when Periodic Scan is selected)The scanning cycle for sensitive columns, supporting Week or Month. That is, automatically perform scanning on which day(s) of the week or which date(s) of the month, supporting multiple selections.
    Selecting Every day means executing the scan once every day.
    Launch TimeThe start time of the sensitive column scanning.
    • Single Scan scenario: Optional. You need to select the date and time. If not specified, it means the scan starts immediately after configuration is completed.
    • Periodic Scan scenario: Required. You need to select the time point.
    Scan Results Activate ImmediatelyConfiguration of the effective time of sensitive columns. Choose whether the scanned sensitive columns take effect immediately after the scan is completed.

  3. Click the Scan Task tab to view the ongoing scanning tasks. On this page, you can perform the following operations:

    • Adding Sensitive Columns: If you selected No in the configuration item of Scan Results Activate Immediately, you need to manually select the newly added sensitive columns from the scan results. Click the number below Adding Sensitive Columns, in the pop-up window, manually select the checkboxes on the left of the sensitive columns that need to take effect, and then click Implementing Identified Results or Marked As Insensitive.

    • Terminate Scan: For tasks with Status as Scanning, you can click Terminate Scan in the Actions column on the right side to end the scanning work in advance.

    • Rescan: If a scan task fails, you can restart the scan by clicking Rescan in the Actions column on the right side of the target task.

Manually Add Sensitive Columns to Data Sources​

If the system fails to detect the sensitive columns you require, you can add them manually.

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.

  1. In the Datasource tab, the system automatically lists all data sources supporting sensitive data configuration. Locate your target data source and click Details under the Actions column.

  2. On the Details page, click the Select dropdown, and select All from the menu.

    image-20250402153223898

  3. Locate the target column and click Configure under the Actions column. Configure:

    • Data Type
    • Masking Algorithm
    • Sensitivity Level Then click OK.
    tip

    For large column lists:

    1. Filter by database/table first
    2. Use column name search for quick navigation

Managing Sensitive Columns​

For sensitive columns that have been added, adjustments can be made to Data Type, Masking Algorithm, and Sensitivity Level.

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.

  1. There are two ways to manage sensitive columns: one is at the data source level, and the other is at the sensitive column level.

    • At the data source level: In the Datasource tab, find the target data source and click Details under the Actions column on its right side.

    • At the sensitive column level: Click the Sensitive Column tab. The system automatically lists all sensitive columns that have been added. You can quickly search by sensitivity level, data source name, database name, table name, and column name.

  2. Find the target sensitive column, click the dropdown icon of the item to be adjusted on its right side, and then select new options in the dropdown menu.

Canceling Periodic Scanning of Sensitive Columns​

For sensitive column scanning tasks that are executed periodically, the system will automatically execute them in a loop based on the time configured by the user. If you need to stop this periodic scanning behavior, you can cancel it manually.

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.
  1. In the Datasource tab, find the target data source, click Cancel Scan under the Actions column on its right side more.
  2. In the pop-up window, click OK.

Terminate Pending Scan Tasks​

Scan tasks enter Pending Exec status after configuration in Datasource tab with scheduled execution time. These can be canceled.

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.

  1. In the Scan Task tab, find the scan task in Pending Exec status, click Terminate Scan under Actions.

  2. Confirm by clicking Terminate in the dialog.

Viewing the Sensitive Data Dashboard​

System administrator users can view the sensitive data dashboard to easily understand the overall status of sensitive data within the current organization.

Interface Description

sensitive_chart

  • Sensitive Data Protection: The total number of data sources supporting sensitive data within the current organization, and the status of sensitive data activation.
  • Datasources: The total number of data sources that have enabled the sensitive data feature, and the sensitivity level of each data source (for example, in the above figure, if Datasources shows 1, and S3 shows 1, it means there is 1 data source that has enabled the sensitive data feature, and the highest level of sensitive columns in that data source is S3).
  • Table Amount: The total number of tables that have enabled the sensitive data feature, and the sensitivity level of each table (for example, in the above figure, if Table Amount shows 1, and S3 shows 1, it means there is 1 table that has enabled the sensitive data feature, and the highest level of sensitive columns in that table is S3).
  • Sensitive Columns: The total number of sensitive columns within the current organization, and the sensitivity levels corresponding to these sensitive columns.
  • Sensitive Data Acess: Displays the number of times sensitive data has been accessed, and the users who have accessed sensitive data.

Operating Steps

  1. Log in to the NineData Console.
  2. In the left navigation bar, click Datasource>Sensitive Data.
  1. Click the Dashboard tab to view.