- StreamNative Platform
- Administer
Work with sink connectors using StreamNative Console
StreamNative Platform enables you to create, update, clone and delete sink connectors. In addition, you can also use the pulsarctl CLI tool to configure and manage sink connectors. For details, see pulsarctl command reference.
Currently, StreamNative Platform supports the following sink connectors.
- AMQP1_0 sink connector
- AWS Lambda sink connector
- AWS SQS sink connector
- Cloud Storage sink connector
- Elasticsearch sink connector
- Google BigQuery sink connector
- Kinesis sink connector
- Data generator sink connector (for internal testing purposes only)
Create a sink connector
This section describes how to create a sink connector.
Prerequisites
This section lists prerequisites for creating supported sink connectors on StreamNative Console and for sink connectors connecting to external systems
Prerequisites for creating an AMQP1_0 sink connector on StreamNative Console
- Log in to the StreamNative Console.
- Grant the admin role or service account with the source or sink permission. For details, see authenticate service accounts.
Prerequisites for an AMQP1_0 sink connector connecting to external systems
- Enable AMQP 1.0-compliant broker service, like RabbitMQ, for remote access.
- Enable proper access credentials for remote connection.
Steps
To create a sink connector, follow these steps.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sinks tab.
Select the target sink connector from the drop-down list.
Configure the sink connector.
Field Description Sink Type (Read-only) the type of the sink connector. Sink Name Enter a name for the sink connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-). Input Topic Configure the input topic for the sink connector.
- Tenant: select the tenant from which you want to export messages.
- Namespace: select the namespace from which you want to export messages.
- Topic: select the topic from which you want to export messages.Replicas Configure the number of Pulsar instances to run the sink connector. Enable auto scaling Enable or disable auto-scaling. If it is enabled, you should set the maximum number of Pulsar instances to run the sink connector. The maximum number of Pulsar instances must be greater than the value of the Replicas
and equal to or smaller than10
.Protocol Enter the protocol. amqps
is the default setting.Host Enter the AMQP service host. Port Enter the AMQP service port. Username Enter the username to authenticate to the AMQP1_0 sink connector. Password Enter the password to authenticate to the AMQP1_0 sink connector. Queue Enter the name of the queue that messages should be read from or written to. Click SUBMIT.
Update a sink connector
You can update some fields to make changes to the configuration of a sink connector after it is created.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sinks tab.
Click the name of the target sink connector to check information about the sink connector.
Make the needed updates.
- Configuration:
- Sink Name: update the sink connector name.
- Input Topic:
- Tenant: select the tenant from which you want to export messages.
- Namespace: select the namespace from which you want to export messages.
- Topic: select the topic from which you want to export messages.
- Replicas: update the number of Pulsar instances to run the sink connector.
- Auto-scaling: when auto-scaling is enabled, update the maximum number of Pulsar instances to run the sink connector.
- Configuration:
Click SUBMIT.
Delete a sink connector
To delete a sink connector, follow these steps.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sinks tab.
Click Delete in the Action column.
Enter the sink connector name and then click Confirm.