Work with source connectors using StreamNative Console

StreamNative Platform enables you to create, update, clone and delete source connectors. In addition, you can also configure and manage connectors by using the pulsarctl CLI tool. For details, see pulsarctl command reference.

Currently, StreamNative Platform supports the following source connectors:

• AMQP1_0 source connector
• AWS SQS source connector
• Debezium Microsoft SQL Server source connector
• Debezium MongoDB source connector
• Debezium MySQL source connector
• Debezium PostgreSQL source connector
• Kafka source connector
• Kinesis source connector
• Data generator source connector ( for internal testing purposes only)

Create a source connector

Before creating a source connector, you need to meet the following prerequisites.

Prerequisites

This section lists prerequisites for creating supported source connectors on StreamNative Console and for source connectors connecting to external systems.

Steps

To create a source connector, follow these steps.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sources tab.

   

3. Select the target source connector from the drop-down list.

   

4. Configure the source connector.

   Select a tabAMQP1_0AWS SQSDebezium Microsoft SQL ServerDebezium MongoDBDebezium MySQLDebezium PostgreSQLKafkaKinesis

   Field	Description
   Source Type	(Read-only) the type of the source connector.
   Source Name	Enter a name for the source connector. The text string supports lowercase characters, numeric characters, and the special character hyphen (-).
   Output Topic	Configure the output topic for the source connector. - Tenant: select the tenant from which you want to ingest messages. - Namespace: select the namespace from which you want to ingest messages. - Topic: select the topic from which you want to ingest the messages.
   Replicas	Set the number of instances for running the source connector.
   Enable auto scaling	Enable or disable auto-scaling. When enabled, you need to set the maximum number of Pulsar instances to run the source connector. The maximum number of Pulsar instances must be greater than the value of the Replicas and equal to or smaller than 10.
   Protocol	Enter the protocol. amqps is the default setting.
   Host	Enter the AMQP service host.
   Port	Enter the AMQP service port.
   Username	Enter the username to authenticate to the AMQP1_0 source connector.
   Password	Enter the password to authenticate to the AMQP1_0 source connector.
   Queue	Enter the name of the queue that messages should be read from or written to.

5. Click SUBMIT.

Update a source connector

You can make changes to the configuration of a source connector after it is created.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sources tab.

3. Click the name of the target source connector to check information about the source connector.

4. Make the needed updates.

   • Configuration:

     • Sink Name: update the source connector name.

   • Output Topic

     • Tenant: select the tenant from which you want to ingest messages.
     • Namespace: select the namespace from which you want to ingest messages.
     • Topic: select the topic from which you want to ingest the messages.

   • Replicas: update the number of instances for running the source connector.

   • Auto-scaling: when auto-scaling is enabled, update the maximum number of Pulsar instances to run the source connector.

5. Click SUBMIT.

Delete a source connector

To delete a source connector, follow these steps.

1. On the left navigation pane, under RESOURCES, click Connector.

2. Select the Created Sources tab.

3. Click Delete in the Action column.

4. Enter the source connector name and then click Confirm.