Set up Your Environment

This section introduces how to set up a new service account to run kafka connectors. To perform the following operations, you need to be the cluster administrator beforehand.

Create a service account for Pulsar users

On the left navigation pane of StreamNative Cloud Console, click Service Accounts.
Click Create Service Account.
Enter a name for the service account, and then click Confirm.

You may see the Role xxxx cannot access topic public/__kafka_connect/__kafka_connect_offset_storage exception when you create a connector, this is a known issue which uses a wrong namespace public/__kafka_connect to create the offset storage topic, after v3.3.1.5, this issue has been fixed, and will use namespace public/default instead.You can create a Super Admin service account or create the public/__kafka_connect namespace and grant produce permission to this service account you created as the workaround.

Authorize the service account

To make the service account work, you need to make the service account granted with proper permissions (packages, produce, and consume). To grant the service account permissions on the namespace level, follow these steps:

Cloud Console

Navigate to the Namespace Dashboard page by switching to the namespace workspace.
On the Namespace Dashboard page, click Configuration on the left navigation pane.
On the Namespace configuration page, click ADD ROLE, and select the service account that you want to authorize.
On the drop-down menu below the service account, select the proper permissions to assign to the service account. There are six permissions in total:

consume: allow the service account to consume messages.
produce: allow the service account to publish messages.
functions: allow the service account to submit and manage functions.
sinks: allow the service account to create and manage sink connectors.
sources: allow the service account to create and manage source connectors.
packages: allow the service account to upload and manage pulsar packages. If you want to submit a customized function/connector, then you will need to upload the function/connector’s JAR/NAR/Python file first, which requires the packages permission.

Grant access to the service account

To grant the underlying infrastructure with access to the newly created service account’s OAuth2 key file, you need to create a service account binding via UI. Go to the Service Accounts tab and choose the service account you want to use for running the connector. Clicking on the right button and there will be a Edit service account bindings option.

Click the Edit service account bindings, choose the desired pool member and confirm.

You can also enable the Enable IAM Role Creation option to create a separate IAM role for the service account. Now your connector is ready to use the service account in StreamNative environments.

(Optional) Create a separate IAM role for the service account

StreamNative’s I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors) run as cloud-native workloads on AWS, GCP, and Azure infrastructures. Use the cloud providers’ native IAM (Identity and Access Management) services to control access to infrastructure resources such as S3, GCS, and Azure Blob Storage. This removes the need for password-based authentication for the service accounts that run connectors in StreamNative Cloud. By default, StreamNative uses a single service account per PulsarCluster for all I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors) to access underlying infrastructure resources. This means all I/O Components in the same cluster share one service account and the same permissions.

Using a single service account for all I/O components means all functions and connectors share the same permissions. If one component is compromised, it could access resources intended for other components.So it’s better to leave the default service account with no permissions and use it for running IO components that do not require access to external resources.And create separate service accounts with the minimum required permissions for each IO component that need to access external resources.

To enhance security and improve isolation, you can create a separate IAM role for each service account used to run connectors. This let you grant only the permissions each service account needs. Create a separate IAM role for your service account:

Snctl
Console

This feature is available in snctl v1.3.0 or later.

Get the PoolMember name and namespace from the PulsarCluster
snctl get pulsarcluster <cluster-id> -o yaml
In the output, find the poolMemberRef block, which looks like:
poolMemberRef: name: <pool-member-name> namespace: <namespace>
Multiple clusters may be located in the same PoolMember. You do not need to create separate IAM roles for each cluster within the same PoolMember.

Create a new ServiceAccountBinding that binds the service account to the PoolMember

snctl create serviceaccountbinding <binding-name> \
  --pool-member <pool-member-namespace>/<pool-member-name> \
  --enable-iam-account-creation \
  --service-account-name <service-account-name>

AWS only: You can specify one or more AWS Assume Role ARNs that can be assumed by the IAM role created for the service account (repeat the flag for each ARN):

snctl create serviceaccountbinding <binding-name> \
  --pool-member <pool-member-namespace>/<pool-member-name> \
  --enable-iam-account-creation \
  --service-account-name <service-account-name> \
  --aws-assume-role-arns <arn1> \
  --aws-assume-role-arns <arn2>

The IAM role created by StreamNative will include permissions to call sts:AssumeRole on arn1 and arn2. You must still add a trust policy on arn1 and arn2 to allow the newly created role to assume them, an example likes below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::[aws-account-id]:role/StreamNative/sncloud-role/authorization.streamnative.io/iamaccounts/IamAccount-[org-id]-sab-[binding-name]"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

(Optional) Update an existing ServiceAccountBinding to create the IAM role
snctl edit serviceaccountbinding <binding-name>

Verify the IAM role was created successfully

snctl get serviceaccountbinding <binding-name> -o yaml

Expected output (example):

apiVersion: cloud.streamnative.io/v1alpha1
kind: ServiceAccountBinding
metadata:
  creationTimestamp: "2025-06-25T08:45:35Z"
  finalizers:
  - serviceaccountbinding.finalizers.cloud.streamnative.io
  generation: 1
  name: test-admin
  namespace: o-lftqu
  ownerReferences:
  - apiVersion: cloud.streamnative.io/v1alpha1
    kind: ServiceAccount
    name: admin
    uid: 4ef639aa-6278-4863-9a23-f1da50cea448
  resourceVersion: "54707713"
  uid: 918d40ad-551d-416b-a2ae-d41548d6608e
spec:
  enableIamRoleCreation: true
  poolMemberRef:
    name: azure-eastus-zephyr
    namespace: streamnative
  serviceAccountName: admin
status:
  conditions:
  - lastTransitionTime: "2025-06-25T08:45:35Z"
    status: "True"
    type: IAMAccountReady
  - lastTransitionTime: "2025-06-25T08:45:35Z"
    status: "True"
    type: ServiceAccountReady
  - lastTransitionTime: "2025-07-16T13:56:17Z"
    status: "True"
    type: ResourceExists
  - lastTransitionTime: "2025-07-16T13:56:17Z"
    status: "True"
    type: PoolMemberReady
  - lastTransitionTime: "2025-07-16T13:56:17Z"
    reason: AllConditionStatusTrue
    status: "True"
    type: Ready

In the output, the status.conditions array should include a condition with type: IAMAccountReady and status: "True", indicating the IAM role was created successfully.

Use the service account when creating I/O components

You can now select this service account in Console (or use its API key with the CLI) when creating I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors). The components will inherit the permissions granted to the IAM role created in the previous step.

You can also create a separate IAM role for your service account via StreamNative Cloud Console. Just enable the Enable IAM Role Creation option when creating or editing a service account binding.

AWS only: You can specify one or more AWS Assume Role ARNs that can be assumed by the IAM role created for the service account (use one line for each ARN)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::[aws-account-id]:role/StreamNative/sncloud-role/authorization.streamnative.io/iamaccounts/IamAccount-[org-id]-sab-[binding-name]"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

In Azure, a Managed Identity sab-[binding-name]-[org-id] is created;In AWS, an IAM role role/StreamNative/sncloud-role/authorization.streamnative.io/iamaccounts/IamAccount-[org-id]-sab-[binding-name] is created;In GCP, a service account with display name: IamAccount/[org-id]/sab-[binding-name] is created.

Set up client tools

Using StreamNative Cloud CLI tool `snctl`

Starting from v1.0.0 of the StreamNative Cloud CLI tool snctl, it supports to manage Kafka Connectors running on StreamNative Cloud. Follow the steps below to set up snctl:

Use snctl config set --organization $ORG to set your StreamNative Cloud organization.
Use snctl context use to select your target StreamNative Cloud cluster interactively.
To send Kafka Connect admin requests as a service account, use snctl kafka admin connect --as-service-account $SERVICE_ACCOUNT_NAME ... or snctl kafka admin connect --use-service-account ....
To run a Kafka connector as a service account, use --sn-service-account $SERVICE_ACCOUNT_NAME on snctl kafka admin connect apply or snctl kafka admin connect patch connector. To select the runtime service account interactively, use --use-sn-service-account.

Verify the setup using snctl kafka admin connect info, it should print something like below.

> snctl kafka admin connect info
URL:               https://${KAFKA-SERVICE-URL}/admin/kafkaconnect/
Version:           3.7.0
Commit:            839b886f9b732b15
Kafka Cluster ID:  connect

--as-service-account and --sn-service-account set different identities.Use --as-service-account to send the request with the specified service account’s credentials. That same service account also becomes the runtime identity of the connector. Therefore, the service account must have permissions to create or update connectors, download packages, and produce or consume messages.Use --sn-service-account to keep the request authenticated as the current caller, but run the connector with the specified service account as its runtime identity. In this case, the caller must have permission to create or update the connector and use the selected service account (see account-admin). The runtime service account only needs the permissions required by the connector itself, such as producing or consuming messages and downloading packages.You can use --as-service-account on Pulsar clusters of any supported version. The --sn-service-account and --use-sn-service-account flags require Pulsar 4.0.x version 4.0.10.6 or later, or Pulsar 4.2.x version 4.2.1.4 or later.

Using kcctl

We have tested with the kcctl to manage Kafka Connectors, you can also try other CLI tools you like. Follow the steps below to set up it:

Create an apikey from the service account you created. You can follow the instructions to create an API key for the service account you choose to use.

Set up the kcctl with the apikey.

kcctl config set-context --bootstrap-servers ${KAFKA-SERVICE-URL}:9093 --cluster=https://${KAFKA-SERVICE-URL}/admin/kafkaconnect/ --username public/default --password "token:${APIKEY}" ${NAME}
kcctl config use-context ${NAME}

The KAFKA-SERVICE-URL is the endpoint of the Kafka service, you can find it in the StreamNative Cloud Console.

Verify the setup using kcctl info, it should print something like below.

> kcctl info
URL:               https://${KAFKA-SERVICE-URL}/admin/kafkaconnect/
Version:           3.7.0
Commit:            839b886f9b732b15
Kafka Cluster ID:  connect

Get Started

Clusters

Data Streams

Process

Connect

Lakehouse

Governance

Pulsar Clients

MQTT Clients

Tools

Pulsar Changelogs

Create a service account for Pulsar users

Authorize the service account

Grant access to the service account

(Optional) Create a separate IAM role for the service account

Set up client tools

Using StreamNative Cloud CLI tool `snctl`

Using kcctl

What’s next?

​Create a service account for Pulsar users

​Authorize the service account

​Grant access to the service account

​(Optional) Create a separate IAM role for the service account

​Set up client tools

​Using StreamNative Cloud CLI tool snctl

​Using kcctl

​What’s next?

Create a service account for Pulsar users

Authorize the service account

Grant access to the service account

(Optional) Create a separate IAM role for the service account

Set up client tools

Using StreamNative Cloud CLI tool `snctl`

Using kcctl

What’s next?