Create a service account for Pulsar users
- On the left navigation pane of StreamNative Cloud Console, click Service Accounts.
- Click Create Service Account.
- Enter a name for the service account, and then click Confirm.
You may see the
Role xxxx cannot access topic public/__kafka_connect/__kafka_connect_offset_storage
exception when you create a connector, this is a known issue and will be fixed in v3.3.1.5.You can create a Super Admin service account or create the public/__kafka_connect
namespace and grant produce
permission to this service account you created as the workaround.Authorize the service account
To grant the service account permissions on the namespace level, follow these steps:- Navigate to the Namespace Dashboard page by switching to the namespace workspace.
- On the Namespace Dashboard page, click Configuration on the left navigation pane.
- On the Namespace configuration page, click ADD ROLE, and select the service account that you want to authorize.
- On the drop-down menu below the service account, select the proper permissions to assign to the service account. There are six permissions in total:
-
consume
: allow the service account to consume messages. -
produce
: allow the service account to publish messages. -
functions
: allow the service account to submit and manage functions. -
sinks
: allow the service account to create and manage sink connectors. -
sources
: allow the service account to create and manage source connectors. -
packages
: allow the service account to upload and manage pulsar packages. If you want to submit a customized function/connector, then you will need to upload the function/connector’s JAR/NAR/Python file first, which requires thepackages
permission.
Grant access to the service account
To grant the underlying infrastructure with access to the newly created service account’s OAuth2 key file, you need to create a service account binding via UI. Go to theService Accounts
tab and choose the service account you want to use for running the connector. Clicking on the right button and there willbe a Edit service account bindings
option.

Edit service account bindings
, choose the desired pool member and confirm.

(Optional) Create a separate IAM role for the service account
StreamNative’s I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors) run as cloud-native workloads on AWS, GCP, and Azure infrastructures. You can use the cloud providers’ native IAM (Identity and Access Management) services to control access to infrastructure resources such as S3, GCS, and Azure Blob Storage. This removes the need for password-based authentication for the service accounts that run connectors in StreamNative Cloud. By default, StreamNative uses a single service account perPulsarCluster
for all I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors) to access underlying infrastructure resources. This means all I/O Components in the same cluster share one service account and the same permissions.
Using a single service account for all I/O components means all functions and connectors share the same permissions. If one component is compromised, it could access resources intended for other components.So it’s better to leave the default service account with no permissions and use it for running IO components that do not require access to external resources.And create separate service accounts with the minimum required permissions for each IO component that need to access external resources.
This feature is available in snctl v1.3.0 or later.Support in Console will be added soon.
-
Get the
PoolMember
name and namespace from thePulsarCluster
In the output, find thepoolMemberRef
block, which looks like:Multiple clusters may be located in the samePoolMember
. You do not need to create separate IAM roles for each cluster within the samePoolMember
. -
Create a new
ServiceAccountBinding
that binds the service account to thePoolMember
AWS only: You can specify one or more AWS IAM role ARNs that can be assumed by the IAM role created for the service account (repeat the flag for each ARN):The IAM role created by StreamNative will include permissions to callsts:AssumeRole
onarn1
andarn2
. You must still add a trust policy onarn1
andarn2
to allow the newly created role to assume them. -
(Optional) Update an existing
ServiceAccountBinding
to create the IAM role -
Verify the IAM role was created successfully
Expected output (example):In the output, the
status.conditions
array should include a condition withtype: IAMAccountReady
andstatus: "True"
, indicating the IAM role was created successfully.
In Azure, a Managed Identitysab-[binding-name]-[org-id]
is created;In AWS, an IAM rolerole/StreamNative/sncloud-role/authorization.streamnative.io/iamaccounts/IamAccount-[org-id]-sab-[binding-name]
is created;In GCP, a service account with display name:IamAccount/[org-id]/sab-[binding-name]
is created. -
Use the service account when creating I/O components
You can now select this service account in Console (or use its API key with the CLI) when creating I/O components (Pulsar Functions, Pulsar Connectors, and Kafka Connectors). The components will inherit the permissions granted to the IAM role created in the previous step.
Set up client tools
Using StreamNative Cloud CLI tool snctl
Starting from v1.0.0 of the StreamNative Cloud CLI tool snctl
, it supports to manage Kafka Connectors running on StreamNative Cloud.
Follow the steps below to set up snctl
:
-
use
snctl config set --organization $ORG
to your StreamNative Cloud organization. -
use
snctl context use
to interactivly select your target StreamNative Cloud cluster. -
use
snctl kafka admin connect --as-service-account $SERVICE_ACCOUNT_NAME ...
orsnctl kafka admin connect --use-service-account ...
to send Kafka Connect admin requests with selected Service Account. -
Verify the setup using
snctl kafka admin connect info
, it should print something like below.
Using kcctl
We have tested with the kcctl to manage Kafka Connectors, you can also try other CLI tools you like. Follow the steps below to set up it:- Create an apikey from the service account you created. You can follow the instructions to create an API key for the service account you choose to use.
-
Set up the kcctl with the apikey.
The KAFKA-SERVICE-URL is the endpoint of the Kafka service, you can find it in the StreamNative Cloud Console.
-
Verify the setup using
kcctl info
, it should print something like below.