streamnative.io
Help Center
Login
  1. StreamNative CLI

StreamNative CLI Tutorial

This tutorial demonstrates how to use the StreamNative command-line tools to deploy a Serverless cluster and manage Pulsar resources within it. We will primarily focus on the modern, unified approach using StreamNative Cloud CLI (snctl) for both cloud infrastructure and Pulsar resource management/interaction. We will also show alternative methods using the traditional pulsarctl and pulsar-client tools.

This tutorial covers:

  1. Provisioning a Serverless Instance (snctl).
  2. Provisioning a Serverless Cluster (snctl).
  3. Provisioning an Application Service Account (snctl).
  4. Creating an API Key for the Service Account (snctl).
  5. Method 1 (Unified snctl):
    • Configuring snctl Service Context.
    • Creating Pulsar resources (tenant, namespace, topic) using snctl pulsar admin.
    • Granting permissions using snctl pulsar admin.
    • Producing/Consuming messages using snctl pulsar client.
  6. Method 2 (Traditional Tools - Alternatives):
    • Configuring pulsarctl context using the API Key.
    • Creating Pulsar resources using pulsarctl.
    • Granting permissions using pulsarctl.
    • Producing/Consuming messages using pulsar-client configured with the API Key.

0. Prerequisites

Install Required CLIs

Make sure you have the following installed:

  • snctl (v1.0.0+ recommended)
  • pulsarctl (Needed for the alternative method)
  • An Apache Pulsar distribution (e.g., from Pulsar Downloads) for the pulsar-client alternative method.

Create a new directory for the tutorial

Create a new directory for the tutorial.

mkdir snctl-getting-started && cd snctl-getting-started

Create and Activate a Super-Admin Service Account

You need to create a service account with Super Admin access in StreamNative Cloud Console. Let's name it snctl-super-admin. After creating the service account, download and save its OAuth2 credentials file (e.g., snctl-super-admin-credentials.json).

Activate this service account for snctl. This identity will be used for provisioning cloud resources (instances, clusters, other service accounts). Make sure to replace /path/to/snctl-super-admin-credentials.json with the actual path.

# Activate the super-admin service account
snctl auth activate-service-account --key-file=/path/to/snctl-super-admin-credentials.json

After the service account is activated, you should see a similar message:

Logged in as snctl-super-admin@<your-org-id>.auth.streamnative.cloud.
Welcome to StreamNative Cloud!

Set the target organization as the default organization for snctl to avoid specifying -n or -O repeatedly. Replace <your-org-id> with your actual organization ID.

snctl config set --organization <your-org-id>

1. Provision a Serverless Instance

Edit a file named 001-instance.yaml with the following content:

apiVersion: cloud.streamnative.io/v1alpha1
kind: PulsarInstance
metadata:
  name: <your-instance-name>
  namespace: <your-org-id>
spec:
  availabilityMode: regional
  poolRef:
    name: shared-gcp
    namespace: streamnative
  type: serverless

This yaml file defines a Serverless Instance running in GCP with regional availability mode. Replace the following placeholders with your actual values:

  • <your-instance-name>: The name of the Serverless Instance.
  • <your-org-id>: The organization ID.

Run the following command to provision the Serverless Instance:

snctl create -f 001-instance.yaml

You should see the following message:

pulsarinstance.cloud.streamnative.io/<your-instance-name> created

Query the instance to verify the instance is created.

snctl get PulsarInstance <your-instance-name> -o yaml

You will be able to see a similar status block of this instance in the output:

status:
  auth:
    oauth2:
      audience: urn:sn:pulsar:<your-org-id>:<your-instance-name>
      issuerURL: https://auth.streamnative.cloud/
    type: oauth2
  conditions:
    - lastTransitionTime: '...'
      message: a payment method is not required because discount is active
      reason: HasActiveDiscount
      status: 'True'
      type: SubscriptionReady
    - lastTransitionTime: '...'
      reason: Created
      status: 'True'
      type: ResourceServerReady
    - lastTransitionTime: '...'
      reason: Created
      status: 'True'
      type: ServiceAccountReady
    - lastTransitionTime: '...'
      reason: AllConditionStatusTrue
      status: 'True'
      type: Ready

When all the conditions are True, the instance is ready.

2. Provision a Serverless Cluster

Edit a file named 002-cluster.yaml with the following content:

apiVersion: cloud.streamnative.io/v1alpha1
kind: PulsarCluster
metadata:
  namespace: <your-org-id>
spec:
  # currently the `broker` section is still required despite the fact that
  # settings are not used by serverless
  broker:
    replicas: 2
    resources:
      cpu: '1'
      memory: 4Gi
  displayName: serverless-cluster
  instanceName: <your-instance-name>
  location: us-central1

This yaml file defines a Serverless Cluster in us-central1 region. Replace the following placeholders with your actual values:

  • <your-instance-name>: The name of the Serverless Instance.
  • <your-org-id>: The organization ID.

Run the following command to provision the Serverless Instance:

snctl create -f 002-cluster.yaml

You should see the following message:

pulsarcluster.cloud.streamnative.io/<your-cluster-name> created

Note

Please note the <your-cluster-name> in the output message because the cluster name of a Serverless Cluster is generated by StreamNative Cloud. You will need this cluster name in the future steps.

Query the cluster to verify the cluster is created.

snctl get PulsarCluster <your-cluster-name> -o yaml

You will be able to see a similar status block of this cluster in the output:

status:
  broker:
    readyReplicas: 2
    replicas: 2
    updatedReplicas: 2
  conditions:
    - lastTransitionTime: '...'
      reason: Deploy
      status: 'True'
      type: PulsarBrokerReady
    - lastTransitionTime: '...'
      reason: AllConditionStatusTrue
      status: 'True'
      type: Ready
    - lastTransitionTime: '...'
      reason: Ready
      status: 'True'
      type: PulsarInstanceReady
  deploymentType: hosted
  instanceType: serverless

Wait for all the conditions to be True, then the cluster is ready. A Serverless Cluster is usually ready within 1~2 minutes.

3. Provision a Service Account

Edit a file named 003-sa.yaml with the following content:

apiVersion: cloud.streamnative.io/v1alpha1
kind: ServiceAccount
metadata:
  name: <your-service-account-name>
  namespace: <your-org-id>
spec: {}

This yaml file defines a Service Account with a name <your-service-account-name>. Replace the following placeholders with your actual values:

  • <your-service-account-name>: The name of the Service Account.
  • <your-org-id>: The organization ID.

Run the following command to provision the Service Account:

snctl create -f 003-sa.yaml

You should see the following message:

serviceaccount.cloud.streamnative.io/<your-service-account-name> created

Query the service account to verify the service account is created.

snctl get ServiceAccount <your-service-account-name> -o yaml

You will be able to see a similar status block of this service account in the output:

status:
  conditions:
    - lastTransitionTime: '...'
      reason: Provisioned
      status: 'True'
      type: Ready
  privateKeyData: ...
  privateKeyType: TYPE_SN_CREDENTIALS_FILE

Wait until the Ready condition is True, then the service account is ready.

4. Create an API Key for the Service Account (Optional but shown for completeness)

While snctl typically uses OAuth2 via auth activate-service-account or context impersonation (--as-service-account), you might need an API Key for external clients or tools that only support token authentication. This API key will be used in the alternative pulsarctl and pulsar-client method later.

Edit a file named 004-api-key.yaml:

apiVersion: cloud.streamnative.io/v1alpha1
kind: APIKey
metadata:
  name: <your-api-key-name>
  namespace: <your-org-id>
spec:
  description: This is a test api key for <your-service-account-name> in running the snctl tutorial
  instanceName: <your-instance-name>
  serviceAccountName: <your-service-account-name>

This yaml file defines an API Key for the Service Account <your-service-account-name>. Replace the following placeholders with your actual values:

  • <your-api-key-name>: The name of the API Key.
  • <your-service-account-name>: The name of the Service Account.
  • <your-instance-name>: The name of the Serverless Instance.
  • <your-org-id>: The organization ID.

Run the following command to create the API Key:

snctl create -f 004-api-key.yaml

You should see the following message:

apikey.cloud.streamnative.io/<your-api-key-name> created

Query the API Key to verify the API Key is created and optionally retrieve the token.

snctl get apikey <your-api-key-name> -o yaml

You will be able to see a similar status block of this API Key in the output:

status:
  conditions:
    - lastTransitionTime: '...'
      message: ''
      reason: API Key has been provisioned
      status: 'True'
      type: Issued
    - lastTransitionTime: '...'
      message: ''
      reason: API Key is not revoked
      status: 'False'
      type: Revoked
    - lastTransitionTime: '...'
      message: ''
      reason: API Key will never expire
      status: 'False'
      type: Expired
  expiresAt: '1970-01-01T00:00:00Z'
  issuedAt: '...'
  keyId: <key-id>
  token: <token>

Wait until the Issued condition is True, then the API Key is issued and ready to use. You can obtain the token from the token field in the status block.

You can use the following command to obtain the token and export it as an environment variable API_KEY_TOKEN:

export API_KEY_TOKEN=$(snctl get apikey <your-api-key-name> -o jsonpath='{.status.token}')

Method 1: Unified Management with snctl

This section demonstrates using snctl for configuring access, managing Pulsar resources, and interacting with the data plane.

5. Configure snctl Service Context for Pulsar Interaction

snctl uses Service Contexts to manage connections to Pulsar/Kafka clusters. After creating a cluster, snctl usually discovers it automatically. Let's explicitly set the context for the cluster we created to ensure subsequent commands target it correctly.

Set the active context to your newly created cluster. Replace <your-instance-name> and <your-cluster-name> with the actual name from step 2.

snctl context use --pulsar-instance <your-instance-name> --pulsar-cluster <your-cluster-name>

Verify the current context:

snctl context current

Now, verify connectivity by listing tenants. Since we activated the snctl-super-admin service account (in step 0), snctl commands will run as that identity by default.

snctl pulsar admin tenants list

You should see the default tenants:

public
pulsar
sn

This confirms snctl can communicate with the Pulsar cluster's admin endpoint using the super-admin credentials via the active context.

6. Create Pulsar Resources using snctl

Now, let's create the tenant, namespace, and topic for our application sl-app using snctl pulsar admin commands. These commands will use the active context (<your-cluster-name>) and run as the activated super-admin user (snctl-super-admin).

First, create a tenant named sl-app-tenant.

snctl pulsar admin tenants create sl-app-tenant --allowed-clusters <your-cluster-name>

Output:

Tenant "sl-app-tenant" created successfully.

Second, create a namespace named sl-app-ns under the tenant sl-app-tenant.

snctl pulsar admin namespaces create sl-app-tenant/sl-app-ns --clusters <your-cluster-name>

Output:

Namespace "sl-app-tenant/sl-app-ns" created successfully.

Next, create a partitioned topic named sl-app-topic with 4 partitions under the namespace sl-app-tenant/sl-app-ns.

snctl pulsar admin topics create sl-app-tenant/sl-app-ns/sl-app-topic 4

Output:

Create topic persistent://sl-app-tenant/sl-app-ns/sl-app-topic with 4 partitions successfully

Finally, grant the application Service Account <your-service-account-name> (created in step 3) the permission to produce and consume messages within the sl-app-tenant/sl-app-ns namespace. We are still running as snctl-super-admin to grant these permissions. Replace <your-service-account-name> with the name from step 3.

snctl pulsar admin namespaces grant-permission --role <your-service-account-name>@<your-org-id>.auth.streamnative.cloud --actions produce,consume sl-app-tenant/sl-app-ns

Output:

Grant permissions [produce consume] to the client role <your-service-account-name>@<your-org-id>.auth.streamnative.cloud to access the namespace sl-app-tenant/sl-app-ns successfully

7. Produce and consume messages using snctl pulsar client

Now we'll use snctl pulsar client commands to produce and consume messages. Crucially, these actions should be performed as the application service account (<your-service-account-name>) because we granted it the produce/consume permissions, not the super-admin. We use the --as-service-account flag for this, leveraging snctl's ability to impersonate the specified service account (assuming the logged-in super-admin has permission to do so, which is typical).

Produce 10 messages to the topic, acting as the application service account. Replace <your-service-account-name> with the name from step 3.

snctl pulsar client produce --topic sl-app-tenant/sl-app-ns/sl-app-topic \
  --messages "hello sl-app from snctl" \
  --num-times 10 \
  --as-service-account <your-service-account-name>

You should see output indicating successful production, similar to:

Successfully produced 10 message(s) to topic persistent://sl-app-tenant/sl-app-ns/sl-app-topic

(Exact output message may vary)

Consume the 10 messages from the topic, again acting as the application service account. Replace <your-service-account-name> with the name from step 3.

snctl pulsar client consume --topic sl-app-tenant/sl-app-ns/sl-app-topic \
  --subscription-name sl-app-sub \
  --num-messages 10 \
  --initial-position earliest \
  --as-service-account <your-service-account-name>

You should see the 10 messages printed to your console, similar to:

----- got message -----
# ... message details ... content:hello sl-app from snctl
----- got message -----
# ... message details ... content:hello sl-app from snctl
... (10 messages total) ...

Followed by a confirmation like:

Consumed 10 message(s) from topic sl-app-tenant/sl-app-ns/sl-app-topic

(Exact output format may vary)

This demonstrates using snctl for the entire lifecycle using the unified approach.

Method 2: Traditional Management with pulsarctl and pulsar-client (Alternative)

This section demonstrates the alternative approach using the separate pulsarctl tool for admin tasks and the pulsar-client tool for producing/consuming. This method often relies on API Key authentication for simplicity when interacting with StreamNative Cloud clusters via these tools.

Alt 5. Configure pulsarctl Context (Using API Key)

First, get the admin service URL of the cluster by running the following command:

export ADMIN_SERVICE_URL="https://$(snctl get PulsarCluster <your-cluster-name> -o jsonpath='{.spec.serviceEndpoints[0].dnsName}')"
export BROKER_SERVICE_URL="pulsar+ssl://$(snctl get PulsarCluster <your-cluster-name> -o jsonpath='{.spec.serviceEndpoints[1].dnsName}'):6651"

Once you get the ADMIN_SERVICE_URL, you can use the following command to configure pulsarctl to access the cluster we created in the previous steps:

pulsarctl context set -s ${ADMIN_SERVICE_URL} --key-file /path/to/oauth2-credentials-file.json --audience urn:sn:pulsar:<your-org-id>:<your-instance-name> <your-cluster-name>-admin

This command will create a new context named <your-cluster-name>-admin and update the pulsarctl configuration to use the oauth2 credentials of snctl-super-admin to authenticate to the cluster.

You should see the following message:

Context "<your-cluster-name>-admin" created.

You can verify the pulsarctl has been configured properly by running the following command:

pulsarctl context current

You should see the following message:

<your-cluster-name>-admin

Then you can run pulsarctl tenants list to verify if you configured the pulsarctl properly.

pulsarctl tenants list -o yaml

You should be able to see the tenants in the cluster.

- public
- pulsar
- sn

Alt 6. Create Pulsar Resources using pulsarctl

Assume you want to build a sample application sl-app that produces messages to a topic persistent://sl-app-tenant/sl-app-ns/sl-app-topic and consumes messages from the same topic.

First, create a tenant named sl-app-tenant.

pulsarctl tenants create sl-app-tenant --allowed-clusters <your-cluster-name>

Output:

Create tenant sl-app-tenant successfully

Second, create a namespace named sl-app-ns under the tenant sl-app-tenant.

pulsarctl namespaces create sl-app-tenant/sl-app-ns --clusters <your-cluster-name>

Output:

Created sl-app-tenant/sl-app-ns successfully

Next, create a topic named sl-app-topic with 4 partitions under the namespace sl-app-tenant/sl-app-ns.

pulsarctl topics create sl-app-tenant/sl-app-ns/sl-app-topic 4

Output:

Create topic persistent://sl-app-tenant/sl-app-ns/sl-app-topic with 4 partitions successfully

Finally, grant the Service Account <your-service-account-name> the permission to produce and consume messages from the namespace sl-app-tenant/sl-app-ns.

pulsarctl namespaces grant-permission --role <your-service-account-name>@<your-org-id>.auth.streamnative.cloud --actions produce,consume sl-app-tenant/sl-app-ns

Output:

Grant permissions [produce consume] to the client role <your-service-account-name>@<your-org-id>.auth.streamnative.cloud to access the namespace sl-app-tenant/sl-app-ns successfully

Alt 7. Produce and consume messages using pulsar-client

  1. Download the Pulsar distribution from Pulsar Downloads. Assume you have downloaded the Pulsar distribution and extracted the tarball to /path/to/pulsar-dist.

  2. Enter the root directory of the Pulsar distribution:

    cd /path/to/pulsar-dist

  3. Configure the conf/client.conf file:

    • webServiceUrl: Set the webServiceUrl to the ADMIN_SERVICE_URL you obtained in the previous steps.
    • brokerServiceUrl: Set the brokerServiceUrl to the BROKER_SERVICE_URL you obtained in the previous steps.
    • authPlugin: Set the authPlugin to org.apache.pulsar.client.impl.auth.AuthenticationToken.
    • authParams: Set the authParams to be token:<your-api-key>. <your-api-key> is the API Key you obtained in the previous steps.

  4. Produce 10 messages.

    bin/pulsar-client produce -m "hello sl-app" -n 10 sl-app-tenant/sl-app-ns/sl-app-topic

    You should see a similar message in the output:

    10 messages successfully produced

  5. Consume the messages.

    bin/pulsar-client consume -n 10 -p Earliest -s sl-app-sub sl-app-tenant/sl-app-ns/sl-app-topic

    You should see a similar message in the output:

    ----- got message -----
publishTime:[1732951012862], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951012952], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013162], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013095], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013299], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013368], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013436], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013510], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013025], eventTime:[0], key:[null], properties:[], content:hello sl-app
----- got message -----
publishTime:[1732951013229], eventTime:[0], key:[null], properties:[], content:hello sl-app

    You should see a final message in the output:

    10 messages successfully consumed

8. Next Steps

Once you have verified the application works as expected using either method, you can try out more guided tutorials:

9. Clean up

After you finish the tutorial, you can clean up the resources you created in this tutorial by running the following command:

Note

Please note that you can't use snctl delete -f 002-cluster.yaml to delete the cluster because the cluster name is generated by StreamNative Cloud. So you need to delete the cluster using the snctl delete PulsarCluster <your-cluster-name> command.

snctl delete -f 004-api-key.yaml
snctl delete -f 003-sa.yaml
snctl delete PulsarCluster <your-cluster-name>
snctl delete -f 001-instance.yaml
Previous
Overview
Next
Overview