Accessing Kafka Data Source through the Gateway

NineData supports accessing Kafka data sources through a gateway, allowing you to connect to Kafka without providing an external address, even if Kafka is in a private or dedicated network environment.

Prerequisites

• The host where the gateway is deployed must be within the intranet environment of the Kafka data source and must be able to access the Kafka data source smoothly through the intranet.
• The host where the gateway is deployed must have internet access. If your host cannot access the internet and is within a local area network, you can deploy a proxy gateway. For more information, see Using a Proxy Gateway.

Step 1: Deploy the Gateway

This step is applicable to hosts that can access the public network. If your host cannot access the public network and is in the local area network, see Using Proxy Gateway to deploy the gateway.

1. Log in to the NineData console on the host where the gateway needs to be deployed.

2. In the left navigation pane, click >.

3. Click in the upper right corner of the page.

4. Configure the parameters based on the following table.

   Parameter	Description
   	The name of the gateway. For easy searching and management, manual specification is supported.
   	Select the operating system of the host where the gateway needs to be deployed, and the corresponding gateway installation method is displayed. The following operating systems are supported: Windows(x86_64) Linux(x86_64) macOS
   	Select the region closest to the host where the gateway is deployed to obtain relatively fast access speed.
   	No configuration required.

5. Install and configure the gateway on the host of the deployment end based on the installation method displayed below. After the configuration is completed, the connection diagram on the right side of the page changes from to .

6. Click on , select the desired data source in the pop-up window, and proceed to Step 2.

Step 2: Access the Kafka Data Source

1. On the page, configure the parameters based on the table below.

   Parameter	Description
   	Enter the name of the data source. For ease of later retrieval and management, please use a meaningful name.
   	Select .
   	Select the gateway ID deployed in Step 1.
   Broker List	Enter the connection address and port of the Kafka cluster. If Kafka is installed on the local machine: Enter localhost or 127.0.0.1 on the left and the actual access port of Kafka on the right. If Kafka is installed on another host within the intranet: Enter the intranet IP of that host on the left and the actual access port of Kafka on the right. Note: If there are multiple connection addresses, click to add a new record and continue filling in.
   	Select the authentication mode for Kafka. PLAINTEXT: No authentication information is required, and data is transmitted in plain text. This mode is usually suitable for development and testing environments. SASL/PLAIN (default): Authentication is based on SASL and requires a username and password. Data is transmitted in plain text. It is recommended to use this mode over encrypted networks. SASL/SCRAM-SHA-256: Authentication is based on SASL and requires a username and password. The password is hashed using the SHA-256 algorithm, providing higher security. SASL/SCRAM-SHA-512: Authentication is based on SASL and requires a username and password. The password is hashed using the SHA-512 algorithm, providing higher security.
   	Enter the login username for Kafka.
   	Enter the login password for Kafka.
   	Select the region closest to your Kafka host to effectively reduce network latency.
   	Select according to the actual business purpose of the data source as an environment identifier for the data source. The default provides the and environments, and also supports you to create a custom environment. Note: In organization mode, the database environment can also be applied to permission policy management. For example, the default role only supports access to data sources in the environment and cannot access data sources in other environments. For more information, see Managing Roles.

2. After configuring all parameters, click on the right of to test whether the data source can be accessed normally. If is displayed, click to complete adding the data source. If not, please recheck the connection settings until the connection test succeeds.

Appendix: Using a Proxy Gateway

If the host where the gateway needs to be deployed cannot access the public network, a proxy gateway is needed. You need to prepare two hosts in the same LAN that can access each other:

• Host A: A host that can access the public network.
• Host B: The host where the gateway needs to be deployed.

Create a gateway in Host A. The role of this gateway is to act as a proxy so that Host B can connect to the NineData server through the proxy.

Procedure

1. Log in to the NineData console on Host A.

2. In the left navigation bar, click > .

3. Click in the upper right corner of the page.

4. Configure the parameters according to the following table.

   Parameter	Description
   	The name of the gateway, which can be manually specified for easy search and management.
   	Select the operating system of the host where the gateway needs to be deployed, and the corresponding gateway installation method will be displayed. The following operating systems are supported: Windows (x86_64) Linux (x86_64) macOS
   	Select the region closest to the host where the gateway is deployed to obtain relatively fast access speed.
   	No need to configure.

5. Install and configure the gateway in Host A according to the installation method displayed below. After the configuration is completed, the connection diagram on the right side of the page will change from to .

6. Click , and then click in the upper right corner of the page again.

7. In , select the operating system of Host B.

8. Click , select , and click the drop-down box below . Select the gateway created in Host A from the drop-down menu.

   tip

   At this point, the gateway process will automatically open a port on Host A for Host B's access. When the page prompts "xxx has enabled proxy function, port xxx", it means that the proxy gateway has been successfully opened.

9. Install the gateway on Host B according to the installation steps displayed below.

10. Wait for the connection diagram on the right side of the page to change from to .

11. Click to enter Step 2.