Set up Your Environment

This section introduces how to set up a new service account to run kafka connectors. To perform the following operations, you need to be the cluster administrator beforehand.

Create a service account for Pulsar users

On the left navigation pane of StreamNative Cloud Console, click Service Accounts.
Click Create Service Account.
Enter a name for the service account, and then click Confirm.

Note

You may see the Role xxxx cannot access topic public/__kafka_connect/__kafka_connect_offset_storage exception when you create a connector, this is a known issue and will be fixed in v3.3.1.5.

You can create a Super Admin service account or create the public/__kafka_connect namespace and grant produce permission to this service account you created as the workaround.

Authorize the service account

To grant the service account permissions on the namespace level, follow these steps:

Navigate to the Namespace Dashboard page by switching to the namespace workspace.
On the Namespace Dashboard page, click Configuration on the left navigation pane.
On the Namespace configuration page, click ADD ROLE, and select the service account that you want to authorize.
On the drop-down menu below the service account, select the proper permissions to assign to the service account. There are six permissions in total:

consume: allow the service account to consume messages.
produce: allow the service account to publish messages.
functions: allow the service account to submit and manage functions.
sinks: allow the service account to create and manage sink connectors.
sources: allow the service account to create and manage source connectors.
packages: allow the service account to upload and manage pulsar packages. If you want to submit a customized function/connector, then you will need to upload the function/connector’s JAR/NAR/Python file first, which requires the packages permission.

Authorize Service Account

Grant access to the service account

To grant the underlying infrastructure with access to the newly created service account's OAuth2 key file, you need to create a service account binding via UI.

Go to the Service Accounts tab and choose the service account you want to use for running the connector. Clicking on the right button and there willbe a Edit service account bindings option. Binding Service Account step-1

Click the Edit service account bindings, choose the desired pool member and confirm. Binding Service Account step-2

Now your connector is ready to use the service account in StreamNative environments.

Set up client tools

We have tested with the kcctl to manage Kafka Connectors, you can also try other CLI tools you like.

Follow the steps below to set up it:

Create an apikey from the service account you created.
You can follow the instructions to create an API key for the service account you choose to use.

Set up the kcctl with the apikey.

kcctl config set-context --bootstrap-servers ${KAFKA-SERVICE-URL}:9093 --cluster=https://${KAFKA-SERVICE-URL}/admin/kafkaconnect/ --username public/default --password "token:${APIKEY}" ${NAME}
kcctl config use-context ${NAME}

Note

The KAFKA-SERVICE-URL is the endpoint of the Kafka service, you can find it in the StreamNative Cloud Console.

Verify the setup using kcctl info, it should print something like below.

> kcctl info
URL:               https://${KAFKA-SERVICE-URL}/admin/kafkaconnect/
Version:           3.7.0
Commit:            839b886f9b732b15
Kafka Cluster ID:  connect

Create a service account for Pulsar users

Authorize the service account

Grant access to the service account

Set up client tools

What’s next?