- StreamNative Platform
- Administer
Work with source connectors using StreamNative Console
StreamNative Platform enables you to create, update, clone and delete source connectors. In addition, you can also configure and manage connectors by using the pulsarctl CLI tool. For details, see pulsarctl command reference.
Currently, StreamNative Platform supports the following source connectors:
- AMQP1_0 source connector
- AWS SQS source connector
- Debezium Microsoft SQL Server source connector
- Debezium MongoDB source connector
- Debezium MySQL source connector
- Debezium PostgreSQL source connector
- Kafka source connector
- Kinesis source connector
- Data generator source connector ( for internal testing purposes only)
Create a source connector
Before creating a source connector, you need to meet the following prerequisites.
Prerequisites
This section lists prerequisites for creating supported source connectors on StreamNative Console and for source connectors connecting to external systems.
Prerequisites for creating an AMQP1_0 source connector on StreamNative Console
- Log in to the StreamNative Console.
- Grant the admin role or service account with the source or sink permission. For details, see authenticate service accounts.
Prerequisites for an AMQP1_0 source connector connecting to external systems
- Enable AMQP 1.0-compliant broker service, like RabbitMQ, for remote access.
- Enable proper access credentials for remote connection.
Steps
To create a source connector, follow these steps.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sources tab.
Select the target source connector from the drop-down list.
Configure the source connector.
Field Description Source Type (Read-only) the type of the source connector. Source Name Enter a name for the source connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-). Output Topic Configure the output topic for the source connector.
- Tenant: select the tenant from which you want to ingest messages.
- Namespace: select the namespace from which you want to ingest messages.
- Topic: select the topic from which you want to ingest the messages.Replicas Set the number of instances for running the source connector. Enable auto scaling Enable or disable auto-scaling. When enabled, you need to set the maximum number of Pulsar instances to run the source connector. The maximum number of Pulsar instances must be greater than the value of the Replicas
and equal to or smaller than10
.Protocol Enter the protocol. amqps
is the default setting.Host Enter the AMQP service host. Port Enter the AMQP service port. Username Enter the username to authenticate to the AMQP1_0 source connector. Password Enter the password to authenticate to the AMQP1_0 source connector. Queue Enter the name of the queue that messages should be read from or written to. Click SUBMIT.
Update a source connector
You can make changes to the configuration of a source connector after it is created.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sources tab.
Click the name of the target source connector to check information about the source connector.
Make the needed updates.
Configuration:
- Sink Name: update the source connector name.
Output Topic
- Tenant: select the tenant from which you want to ingest messages.
- Namespace: select the namespace from which you want to ingest messages.
- Topic: select the topic from which you want to ingest the messages.
Replicas: update the number of instances for running the source connector.
Auto-scaling: when auto-scaling is enabled, update the maximum number of Pulsar instances to run the source connector.
Click SUBMIT.
Delete a source connector
To delete a source connector, follow these steps.
On the left navigation pane, under RESOURCES, click Connector.
Select the Created Sources tab.
Click Delete in the Action column.
Enter the source connector name and then click Confirm.