Work with sink connectors using StreamNative Console

StreamNative Platform enables you to create, update, clone and delete sink connectors. In addition, you can also use the pulsarctl CLI tool to configure and manage sink connectors. For details, see pulsarctl command reference.

Currently, StreamNative Platform supports the following sink connectors.

• AMQP1_0 sink connector
• AWS Lambda sink connector
• AWS SQS sink connector
• Cloud Storage sink connector
• Elasticsearch sink connector
• Google BigQuery sink connector
• Kinesis sink connector
• Data generator sink connector (for internal testing purposes only)

Create a sink connector

This section describes how to create a sink connector.

Prerequisites

This section lists prerequisites for creating supported sink connectors on StreamNative Console and for sink connectors connecting to external systems

Steps

To create a sink connector, follow these steps.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sinks tab.

   

3. Select the target sink connector from the drop-down list.

   

4. Configure the sink connector.

   Select a tabAMQP1_0AWS LambdaAWS SQSCloud StorageElasticsearchGoogle BigQuery sinkKinesis

   Field	Description
   Sink Type	(Read-only) the type of the sink connector.
   Sink Name	Enter a name for the sink connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-).
   Input Topic	Configure the input topic for the sink connector. - Tenant: select the tenant from which you want to export messages. - Namespace: select the namespace from which you want to export messages. - Topic: select the topic from which you want to export messages.
   Replicas	Configure the number of Pulsar instances to run the sink connector.
   Enable auto scaling	Enable or disable auto-scaling. If it is enabled, you should set the maximum number of Pulsar instances to run the sink connector. The maximum number of Pulsar instances must be greater than the value of the Replicas and equal to or smaller than 10.
   Protocol	Enter the protocol. amqps is the default setting.
   Host	Enter the AMQP service host.
   Port	Enter the AMQP service port.
   Username	Enter the username to authenticate to the AMQP1_0 sink connector.
   Password	Enter the password to authenticate to the AMQP1_0 sink connector.
   Queue	Enter the name of the queue that messages should be read from or written to.

5. Click SUBMIT.

Update a sink connector

You can update some fields to make changes to the configuration of a sink connector after it is created.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sinks tab.

3. Click the name of the target sink connector to check information about the sink connector.

4. Make the needed updates.

   • Configuration:
     • Sink Name: update the sink connector name.
   • Input Topic:
     • Tenant: select the tenant from which you want to export messages.
     • Namespace: select the namespace from which you want to export messages.
     • Topic: select the topic from which you want to export messages.
   • Replicas: update the number of Pulsar instances to run the sink connector.
   • Auto-scaling: when auto-scaling is enabled, update the maximum number of Pulsar instances to run the sink connector.

5. Click SUBMIT.

Delete a sink connector

To delete a sink connector, follow these steps.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sinks tab.

3. Click Delete in the Action column.

4. Enter the sink connector name and then click Confirm.