- StreamNative Cloud
- Connectors
Work with sink connectors
This document describes how to work with sink connectors through StreamNative Cloud Console. In addition, you can also use the pulsarctl CLI tool to configure and manage sink connectors. For details, see pulsarctl command reference.
Currently, StreamNative Cloud supports the following sink connectors.
- AMQP1_0 sink connector
- AWS SQS sink connector
- Cloud Storage sink connector
- Google BigQuery sink connector
- Kinesis sink connector
- Snowflake sink connector
- Data generator sink connector (for internal testing purposes only)
Create sink connectors
This section describes how to create sink connectors.
Prerequisites
This section lists prerequisites for creating supported sink connectors on StreamNative Cloud Console and for sink connectors connecting to external systems
Prerequisites for creating an AMQP1_0 sink connector on StreamNative Cloud Console
- Deploy one Pulsar cluster (2.8.0+) in StreamNative Cloud. For details, see create clusters through StreamNative Cloud Console.
- Log in to the StreamNative Cloud Console or
snctl
CLI tool.
Prerequisites for an AMQP1_0 sink connector connecting to external systems
- Enable AMQP 1.0-compliant broker service, like RabbitMQ, for remote access.
- Enable proper access credentials for remote connection.
Steps
From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click CREATE A SINK. A drop-down menu appears and lists available sink connectors.
Click the icon of the sink connector that you want to create and enter the page where you can configure the sink connector.
Configure the sink connector. See the tables of valid configuration options below.
Click SUBMIT.
Field Description Sink Type (Read-only) the type of the sink connector. Service Account Select the service account for creating the sink connector. Sink Name Enter a name for the sink connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-). Subscription Name Enter the subscription name for the consumers that consume messages from the input topic. Input Topic Configure the input topic for the sink connector.
- Tenant: select the tenant you want to export messages from.
- Namespace: select the namespace you want to export messages from.
- Topic: select the topic you want to export messages from.Replicas Configure the number of Pulsar instances to run the sink connector. Config Configure the sink connector.
- Protocol: enter the protocol.amqp
is the default setting.
- Host: enter the AMQP service host.
- Port: enter the AMQP service port.
- Username: enter the username to authenticate to the AMQP1_0 sink connector.
- Password: enter the password to authenticate to the AMQP1_0 sink connector.
- Queue: enter the name of the queue that messages should be read from or written to.
Update sink connectors
You can update some fields to make changes to the configuration of a sink connector after it is created.
From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click the ellipsis at the end of the row of the connector that you want to update, and then click Edit.
Edit the configuration that you want to change. For details, see create sink connectors.
Click SUBMIT.
Delete sink connectors
From the left navigation pane, under Resources, click Connectors.
Select the Created Sinks tab.
Click the ellipsis at the end of the row of the connector that you want to delete, and then click Delete.
Enter the sink connector name and then click Confirm.