Work with sink connectors

This document describes how to work with sink connectors through StreamNative Cloud Console. In addition, you can also use the pulsarctl CLI tool to configure and manage sink connectors. For details, see pulsarctl command reference.

Currently, StreamNative Cloud supports the following sink connectors.

AMQP1_0 sink connector
AWS SQS sink connector
Cloud Storage sink connector
Google BigQuery sink connector
Kinesis sink connector
Snowflake sink connector
Data generator sink connector (for internal testing purposes only)

Create sink connectors

This section describes how to create sink connectors.

Prerequisites

This section lists prerequisites for creating supported sink connectors on StreamNative Cloud Console and for sink connectors connecting to external systems

Prerequisites for creating an AMQP1_0 sink connector on StreamNative Cloud Console
- Deploy one Pulsar cluster (2.8.0+) in StreamNative Cloud. For details, see create clusters through StreamNative Cloud Console.
- Log in to the StreamNative Cloud Console or snctl CLI tool.
Prerequisites for an AMQP1_0 sink connector connecting to external systems
- Enable AMQP 1.0-compliant broker service, like RabbitMQ, for remote access.
- Enable proper access credentials for remote connection.

Steps

From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click CREATE A SINK. A drop-down menu appears and lists available sink connectors.
Click the icon of the sink connector that you want to create and enter the page where you can configure the sink connector.
Configure the sink connector. See the tables of valid configuration options below.

Click SUBMIT.

Field	Description
Sink Type	(Read-only) the type of the sink connector.
Service Account	Select the service account for creating the sink connector.
Sink Name	Enter a name for the sink connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-).
Subscription Name	Enter the subscription name for the consumers that consume messages from the input topic.
Input Topic	Configure the input topic for the sink connector. - Tenant: select the tenant you want to export messages from. - Namespace: select the namespace you want to export messages from. - Topic: select the topic you want to export messages from.
Replicas	Configure the number of Pulsar instances to run the sink connector.
Config	Configure the sink connector. - Protocol: enter the protocol. `amqp` is the default setting. - Host: enter the AMQP service host. - Port: enter the AMQP service port. - Username: enter the username to authenticate to the AMQP1_0 sink connector. - Password: enter the password to authenticate to the AMQP1_0 sink connector. - Queue: enter the name of the queue that messages should be read from or written to.

Update sink connectors

You can update some fields to make changes to the configuration of a sink connector after it is created.

From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click the ellipsis at the end of the row of the connector that you want to update, and then click Edit.
Edit the configuration that you want to change. For details, see create sink connectors.
Click SUBMIT.

Delete sink connectors

From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click the ellipsis at the end of the row of the connector that you want to delete, and then click Delete.
Enter the sink connector name and then click Confirm.