Skip to main content

Creating Data Sources

NineData supports adding various types of data sources from different environments to the console for unified management. You can use database DevOps, backup and recovery, data replication, and database comparison features on the added data sources.

Feature Introduction​

  • Currently supported cloud vendors: Alibaba Cloud, Tencent Cloud, Huawei Cloud, AWS, Mobile Cloud, Baidu Cloud

  • Currently supported data source types:

    CategoryData Source
    Database
    • Relational Databases: MySQL, SQL Server, PostgreSQL, Oracle, OceanBase for Oracle, OceanBase for MySQL, Db2, Dameng, KingbaseES for PostgreSQL, KingbaseES for Oracle, Klustron, openGauss, Sybase, TiDB, GreatSQL, GBase, Massive Database, GoldenDB
    • NoSQL Databases: MongoDB, Redis, Milvus
    • Data Warehouses: ClickHouse, Doris, SelectDB, Greenplum, StarRocks, SingleStore, Hive
    • Other Data Sources: Elasticsearch, Kafka
    Alibaba CloudRDS MySQL, PolarDB MySQL, AnalyticDB MySQL, RDS SQL Server, RDS PostgreSQL, AnalyticDB PostgreSQL, Cloud Database MongoDB, Cloud Database Redis, Cloud Database Clickhouse, Search and Analysis Service Elasticsearch, Cloud Message Queue Kafka, AnalyticDB PostgreSQL, DataHub, Lindorm
    Tencent cloudCloud Database MySQL, TDSQL-C MySQL, Cloud Database SQL Server, Cloud Database PostgreSQL, Cloud Database MongoDB, Cloud Database Redis, Elasticsearch Service, Message Queue CKafka, TDSQL MySQL, TDSQL PostgreSQL
    Huawei CloudCloud Database RDS for MySQL, Cloud Database GaussDB for MySQL, Cloud Database RDS for SQL Server, Cloud Database RDS for PostgreSQL, Document Database Service DDS, Cloud Database GaussDB for Redis, Cloud Search Service CSS, Distributed Message Service Kafka, Cloud Database Warehouse (DWS), Cloud Database GaussDB
    AWSRDS MySQL, RDS PostgreSQL, RDS MariaDB, RDS SQL Server, RDS Oracle, Aurora MySQL, Aurora PostgreSQL, DocumentDB, MemoryDB for Redis, Redshift, ElastiCache (Redis)
    ECloudRDS for MySQL, MongoDB, Redis
    Baidu CloudCloud Database RDS MySQL, DocDB for MongoDB, Cloud Database Redis, Cloud Native Database GaiaDB, Cloud Database GaiaDB-X, Cloud Vector Database VectorDB, PegaDB

Precautions​

  • To ensure the service can connect smoothly, please first select Access Region on the Create Datasource page, and then add the NineData service IP address located in the position of the figure below to the whitelist of your server. How to add a whitelist in an IDC self-built database, please refer to Appendix II. How to add a whitelist in cloud vendor servers or databases, please refer to the official documents of each cloud vendor.

    server_ip_address

  • Please ensure that you still have free data source quotas, otherwise, you will not be able to enter data sources. You can quickly check the remaining quota in the upper right corner of the NineData console page. check_quota

Adding Self-built Data Sources​

Please click on the hyperlinks of the target data sources in the table below to view detailed operation guides.

Data Source TypeData Source NameHyperlink
Relational DatabasesMySQL Data SourceCreate MySQL Data Source
PostgreSQL Data SourceCreate PostgreSQL Data Source
PolarDB for OracleCreate PolarDB for Oracle Data Source
PolarDB for PostgreSQLCreate PolarDB for PostgreSQL Data Source
PolarDB-XCreate PolarDB-X Data Source
Oracle Data SourceCreate Oracle Data Source
SQL Server Data SourceCreate SQL Server Data Source
Db2 Data SourceCreate Db2 Data Source
TiDB Data SourceCreate TiDB Data Source
OceanBase MySQL Data SourceCreate OceanBase MySQL Data Source
OceanBase Oracle Data SourceCreate OceanBase Oracle Data Source
Dameng Data SourceCreate Dameng Data Source
Kingbase Data SourceCreate Kingbase Data Source
GBase Data SourceCreate GBase Data Source
openGauss Data SourceCreate openGauss Data Source
Sybase Data SourceCreate Sybase Data Source
GreatSQL Data SourceCreate GreatSQL Data Source
MariaDB Data SourceCreate MariaDB Data Source
NoSQLMongoDB Data SourceCreate MongoDB Data Source
Redis Data SourceCreate Redis Data Source
AWS ElastiCache Data SourceCreate AWS ElastiCache Data Source
Milvus|Zilliz Cloud Data SourceCreate AWS ElastiCache Data Source
Data WarehousesClickHouse Data SourceCreate ClickHouse Data Source
Doris Data SourceCreate Doris Data Source
StarRocks Data SourceCreate StarRocks Data Source
Hive Data SourceCreate Hive Data Source
Greenplum Data SourceCreate Greenplum Data Source
SingleStore Data SourceCreate SingleStore Data Source
SelectDB Data SourceCreate SelectDB Data Source
OthersElasticsearch Data SourceCreate Elasticsearch Data Source
Kafka Data SourceCreate Kafka Data Source

Adding Cloud Vendor Data Sources​

  1. Log in to the NineData Console.

  2. On the left navigation pane, click > .

  3. Click  tab, and click  on the page. In the popup window for selecting the data source type, choose （Cloud Vendor Name） > (the type of data source to be added), and configure the parameters based on the table below on the page.
    tip

    If you make a mistake during the operation, you can click the arrow_down icon at the top of the page to make a new selection.

Supported instance types:

  • RDS for MySQL
  • PolarDB for MySQL
  • AnalyticDB for MySQL
  • RDS for SQL Server
  • RDS for PostgreSQL
  • ApsaraDB for MongoDB
  • ApsaraDB for Redis
  • ApsaraDB for ClickHouse
  • Elasticsearch
  • ApsaraMQ for Kafka
  • AnalyticDB for PostgreSQL
  • DataHub
  • Self-managed Datasource

Parameter
Description
NameEnter the name of the data source. To facilitate subsequent search and management, please use meaningful names as much as possible.
ConnectionChoose the access method of the data source.
  • Private Network: Connect through the cloud vendor's private network connection address, suitable for scenarios with high requirements for transmission rate. You need to first connect this private network address to NineData. The connection method is described in Add Private Network Address, and then select the connected PrivateLink below.
  • IP Address: Connect to the data source through the public network address, and enter the public connection address of the instance in Host.
  • Gateway (only supports self-built databases): A secure and fast intranet access method provided by NineData. You need to first connect the host where the data source is located. The connection method is described in Add Gateway. Then select the connected Gateway and Host below, and Host can be written as localhost or intranet IP.
  • SSH Tunnel: Connect to the data source through an SSH tunnel. You also need to provide SSH Host, SSH Authentication Method, SSH Username and other SSH-related information. For detailed configuration steps, please refer to:
Access MethodChoose the access method of Alibaba Cloud instances, which supports access in the form of Instance ID or Host.
  • Instance ID: Access through the instance ID of ECS or cloud database. To choose this option, you also need to select Access Credentials, Access Region, and Instance ID and other parameters. Access Credentials needs to be configured in advance. The configuration method is described in Configure Cloud Vendor Access Credentials.
  • Host: Access through the connection address of the instance. Depending on the Connection, Host is different. For details, please refer to the description of Connection above.
Authentication DBRequired when the data source is MongoDB, specifying the authentication database for the current user.
DB AccountThe username of the data source.
DB PasswordThe password of the data source.
Access RegionOnly required when accessing the data source through the connection address. Choose the region closest to the location of your data source to effectively reduce network latency.
EnvironmentSelect based on the actual business purpose of the data source, as an environmental identifier for the data source. The default provides PROD and DEV environments, and also supports you to create a custom environment.
Note: Under the organization mode, the database environment can also be applied to permission policy management. For example, the default Prod Admin role only supports access to data sources in the PROD environment and cannot access data sources in other environments. For more information, please refer to Manage Roles.
EncryptionWhether to use SSL encryption to access the data source. If the data source enforces SSL encryption connection, this switch must be turned on, otherwise, the connection fails.
Click the switch on the right to turn on or off encrypted transmission. Some data source types can click the > on the left side of Encryption to expand detailed configuration.
Note: Different types of data sources support different options, please refer to the console.
  1. After completing all the configurations, you can click on the right side of Create Datasource and select Connection Test to test if the data source can be accessed successfully. If you see Connection Successfully indicating a successful connection test, you can then proceed to click on Create Datasource to finalize the addition of the data source. If the connection test does not succeed, please review your connection settings and keep testing until the connection is successfully established.

Appendix 1: Creating an Environment​

To meet the needs of different enterprises, in addition to providing default PROD and DEV environments, NineData also supports manually creating custom environments.

  1. Log in to the NineData Console.

  2. In the left navigation bar, click Datasource>Datasource.

  3. Click on the Environment tab. This shows all the environments currently under the user (personal mode) or organization (organization mode). Here you can Edit or Delete custom created environments.

  4. Click on Create Envir. in the upper right corner of the page.

  5. Complete the configuration according to the following table and click OK.

    ParameterDescription
    NameEnter the name of the environment, which can contain 1 to 16 characters. This name serves as the identifier of the environment and needs to have some distinguishing characteristics. Please try to use meaningful names.
    Label ColorChoose the color of the environment label displayed in the console.

Appendix 2: Add NineData's IP address to the database allowlist (self-built database environment)​

When adding a data source located at On-Premise/Other Cloud, the IP address of the NineData service must be added to the database allowlist to allow NineData to provide services.

This section takes MySQL 8.0 as an example to introduce how to add IP allowlist.

  • New account: Log in to the database using the Root account, and create an account for NineData to access and authorize the corresponding permissions with the following command.

    CREATE USER '<account name>'@'<NineData IP address>' IDENTIFIED BY '<access password>';
    GRANT <privilege name> ON <database name>.* TO '<account name>'@'<NineData IP address>';

  • Existing account: Log in to the database using the Root account, and add NineData IP allowlist and grant corresponding permissions to the existing account with the following command.

GRANT <privilege name> ON <database name>.* TO '<account name>'@'<NineData IP address>' IDENTIFIED BY '<access password>';
tip

When adding a MySQL data source, you need to use the account and corresponding password that have added NineData's IP address as mentioned above.

Appendix 3: Add multi-active tag to data sources ​

If the data source serves for multi-node replication tasks (data replication between three or more data sources), in order to prevent circular replication, multi-active flags are required for all data sources participating in the replication, The multi-active tag of each data source must be globally unique.

  1. Log in to NineData Console.

  2. In the navigation pane on the left, click Datasource>Datasource.

  3. Click the target data source ID to open the Data Source Details page.

    1. In the data source details area (including data source name, ID, creator, creation time, etc.), click Show Details.

  4. Find Multi-Active Tag, click the edit icon on the right side of it.

  5. Enter the Multi-Active Tag, and click OK.

    tip
    • Multi-active tag can contain 1~64 characters.
    • The multi-active tag must be globally unique and cannot be duplicated with other multi-active tag.

Appendix 4: Create Tags​

Tags are used for custom grouping in the slow query analysis feature of the Database DevOps module. By configuring tags for specific data sources, you can flexibly group and display slow queries on the dashboard based on business logic, departments, or other requirements. This helps you more precisely identify and address performance bottlenecks.

  1. Log in to the NineData console.

  2. In the left navigation bar, click Datasource > Datasource.

  3. Click the Tag tab and then click Create Tag on the page.

  4. In the pop-up Create Tag dialog, enter the Tag Name and click OK.

    tip
    • The tag name must be globally unique.
    • After creating the tag, you need to bind it to the target data source in the slow query analysis module. For more information, see Slow Query Analysis.

Common problem​

IssuePossible CauseSolution
Error when adding a data source: http connections are not allowed from xxx.xxx.xxx.xxx:xxxxx.NineData service's IP address is not whitelisted on your server.Before adding a data source, make sure to add NineData's service address to your data source whitelist. For more information, please refer to the Precautions section in this article.
Last updated on