streamnative.io
Help Center
Login
  1. StreamNative CLI
  2. Cloud CLI (snctl)

StreamNative CLI (snctl)

StreamNative Cloud provides a command line tool for communicating with StreamNative Cloud's control plane, using the StreamNative Cloud API. This tool is named snctl. This overview includes how to install and configure snctl, covers snctl syntax, describes the command operations, and provides common examples. For details about each command, including all the supported flags and subcommands, see the snctl reference documentation.

Note

To resolve ambiguity with Apache Pulsar's concept of namespaces, particularly when using snctl pulsar subcommands, the flags --namespace (-n) were updated to --organization (-O) beginning in snctl v1.0.0.

Prerequisties

Before moving on to the subsequent steps, ensure you review the following requirements.

Operating systems

The StreamNative CLI is compatible with the following operating systems and architectures only:

  • macOS with 64-bit Intel chips (Darwin AMD64)
  • macOS with Apple chips (Darwin ARM64)
  • Windows with 64-bit Intel or AMD chips (Microsoft Windows AMD64)
  • Linux with 64-bit Intel or AMD chips (Linux AMD64)
  • Linux with 64-bit ARM chips (Linux ARM64)

Network access

When the StreamNative CLI interacts with StreamNative Cloud, it requires network access to the following domains:

  • api.streamnative.cloud
  • auth.streamnative.cloud
  • log.streamnative.cloud

Install snctl

This section describes how to install snctl on Linux, MAC, and Windows Operating System (OS).

You can use the curl command or use Homebrew to install snctl on Linux.

Install snctl with curl command

  1. Execute the following command to download and install the latest snctl.

    bash -c "$(curl -fsSL https://storage.googleapis.com/downloads.streamnative.cloud/snctl/install.sh)"

    If you want to download a specific version, use the flag -v or --version to specify the version. For example, you can use the following command to download snctl v1.0.0.

    bash -c "$(curl -fsSL https://storage.googleapis.com/downloads.streamnative.cloud/snctl/install.sh) -v v1.0.0"

  2. Check whether snctl is installed successfully.

    snctl version --client

Note

When upgrading from snctl v0.x.x to v1.x.x, please run snctl config init again to ensure all newly introduced configuration settings are applied to your local configuration file.

Configure snctl

This section describes how to configure snctl.

Initialize snctl configuration

Before logging in to snctl, you need to configure snctl. You can either use the snctl config init command to initialize snctl with default configurations or you can use the snctl config set command to update the .snctl/config snctl configuration file.

Set snctl configuration

The .snctl/config file contains configurations about snctl, including OAuth2 configurations, service URL for the StreamNative Cloud API, and so on. You can use the snctl config set command to set snctl configurations.

This example sets a target organization as the default organization. Consequently, in this organization, you can perform other operations on StreamNative Cloud resources without specifying the organization name every time.

Note

With the organization option, you need to use the organization id, not the descriptive organization name. To find the organization id, see Cloud Organization ID.

snctl config set --organization <organization_id>

Sign in to an organization

You need to sign in to an organization before executing any snctl commands. You can sign in either as a specific user or as a service account.

Sign in as a user account

This example shows how to sign in as a user account. In this way, snctl is given a token to impersonate the user.

snctl auth login

Output

Logged in as [email protected].
Welcome to StreamNative Cloud!

Subscribe to StreamNative Cloud

Tips

If your organization already has a valid subscription, you can skip this section.

Before you provision a cluster using snctl, you need to set up a subscription to StreamNative Cloud. If you have a legacy cluster, submit a ticket to get assistance with moving your cluster to the updated subscription plan.

Note

  • The snctl create subscription command is available for snctl version 0.14.1 and above.
  • Ensure to add a payment method if you have not done so already. snctl does not support setting up a payment method. You can add a payment method for your organization through StreamNative Cloud Console. For details, see manage billing using StreamNative Cloud Console.

You can use the snctl create subscription command to create a subscription.

snctl create subscription <subscription_name> -n <organization_name> --offer-type <type_of_offer> --offer <offer_name>

Notes:

  • --offer-type: the offer type. Two options are available:
    • private: the private offer that is made to a specific customer. It is a commitment to a minimum amount of spend over a specified time period.
    • public: the public offer that anyone may subscribe to through StreamNative or Marketplace channels. It is a Pay-As-You-Go subscription with usage-based pricing.
  • --offer: the offer name.
    • For a public offer, there is a well-known name (SN2_OD_HOSTED_CLOUD) that snctl uses by default.
    • For a private offer, you will get the offer name from StreamNative sales.

Output

  • You should see the following output if you create a public offer subscription:

    Preparing the subscription, please wait...
Subscription has been activated.

  • You should see the following output if you create a private offer subscription:

    Preparing the subscription, please wait...
To activate your subscription, please pay the initial invoice.
Open the following URL:
https://stripe.com/...
Subscription has been activated.

Use snctl to provision a Pulsar cluster

This section describes how to provision Pulsar clusters through snctl.

Note

In this section, the target organization name is already set in the .snctl/config file. Therefore, you do not need to use the --organization <organization_name> flag to specify the target organization every time when executing snctl commands. For details about how to set snctl configurations, see set snctl configuration.

Create a Pulsar cluster

This section describes how to create a Dedicated cluster through snctl.

  1. Create a Pulsar instance named neo.

    This example shows how to create the neo instance on the AWS cloud platform. To create a Pulsar instance on Google Cloud, set the infrastructure pool name to streamnative/shared. To create a BYOC instance/cluster, the StreamNative team needs to provision an infrastructure pool in your cloud account before you can use it.

    snctl create pulsarinstance neo --pool streamnative/shared-aws

    Output

    pulsarinstance.cloud.streamnative.io/neo created

  2. Find the available locations of a given infrastructure pool to deploy a Pulsar cluster.

    This example shows how to find the available locations for the infrastructure pool streamnative/shared-aws.

    snctl get pooloptions streamnative-shared-aws -o yaml

    You can find the locations shown up in the spec.locations section of the output.

    spec:
  cloudType: aws
  deploymentType: ""
  features:
    AutoScaling: true
    Function: true
    Istio: true
    KOP: true
    MOP: true
    Transaction: true
    WebSocket: true
  locations:
  - location: ap-southeast-2
  - location: eu-central-1
  - location: eu-west-1
  - location: us-east-2

  3. Create a Pulsar cluster named neo-1, consisting of 2 brokers, each using 0.5 CU, and 3 bookies, each using 0.5 SU, to be deployed in us-east-1.

    Note

    The following command is available for snctl version 0.15.0 and above.

    snctl create pulsarcluster neo-1 --instance-name neo \
  --bookie-replicas 3 \
  --storage-unit 0.5 \
  --broker-replicas 3 \
  --compute-unit 0.5 \
  --location us-east-2

    Output

    pulsarcluster.cloud.streamnative.io/neo-1 created

Get the details of a Pulsar cluster

You can use the snctl describe pulsarcluster <cluster_name> command to get details of a Pulsar cluster.

This example gets details of the neo-1 cluster.

snctl describe pulsarcluster neo-1

Output

Cluster neo-1
Name:         neo-1
Namespace:    matrix
Labels:       <none>
Annotations:  <none>
API Version:  cloud.streamnative.io/v1alpha1
Kind:         PulsarCluster
Metadata:
  Creation Timestamp:  2021-01-25T14:48:27Z
  Finalizers:
    pulsarcluster.finalizers.cloud.streamnative.io
  Generation:  2
  Managed Fields:
    API Version:  cloud.streamnative.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        f:bookkeeper:
          .:
          f:replicas:
          f:resourceSpec:
        f:broker:
          f:resourceSpec:
            .:
            f:nodeType:
        f:instanceName:
        f:location:
    Manager:      snctl
    Operation:    Update
    Time:         2021-01-25T14:48:27Z
    API Version:  cloud.streamnative.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        f:broker:
          f:replicas:
    Manager:      kubectl
    Operation:    Update
    Time:         2021-01-29T09:51:58Z
    API Version:  cloud.streamnative.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"pulsarcluster.finalizers.cloud.streamnative.io":
      f:status:
        f:conditions:
          .:
          k:{"type":"BookKeeperReady"}:
            .:
            f:lastTransitionTime:
            f:status:
            f:type:
          k:{"type":"PulsarBrokerReady"}:
            .:
            f:lastTransitionTime:
            f:reason:
            f:status:
            f:type:
          k:{"type":"PulsarProxyReady"}:
            .:
            f:lastTransitionTime:
            f:reason:
            f:status:
            f:type:
          k:{"type":"Ready"}:
            .:
            f:lastTransitionTime:
            f:reason:
            f:status:
            f:type:
          k:{"type":"ZookeeperReady"}:
            .:
            f:lastTransitionTime:
            f:status:
            f:type:
    Manager:         controller-manager
    Operation:       Update
    Time:            2021-02-05T03:16:49Z
  Resource Version:  18740114
  Self Link:         /apis/cloud.streamnative.io/v1alpha1/namespaces/matrix/pulsarclusters/neo-1
  UID:               774c4774-47c7-4913-b9b7-58c47c2f5e43
Spec:
  Bookkeeper:
    Image:     docker.cloudsmith.io/streamnative/cloud-pulsar/pulsar-cloud:3.0.1.6
    Replicas:  3
    Resources:
      Cpu:                1
      Direct Percentage:  0
      Heap Percentage:    0
      Journal Disk:       8G
      Ledger Disk:        64G
      Memory:             4294967296
  Broker:
    Image:     docker.cloudsmith.io/streamnative/cloud-pulsar/pulsar-cloud:3.0.1.6
    Replicas:  3
    Resources:
      Cpu:                1
      Direct Percentage:  0
      Heap Percentage:    0
      Memory:             4294967296
  Instance Name:          neo
  Location:               us-east-1
  Pool Member Ref:
    Name:       aws-use2-production-snci-pool-kid
    Namespace:  streamnative
  Service Endpoints:
    Dns Name:  neo-1.matrix.aws-us-east-1.streamnative.aws.snio.cloud
    Type:      service
Status:
  Bookkeeper:
    Ready Replicas:    3
    Replicas:          3
    Updated Replicas:  3
  Broker:
    Ready Replicas:    2
    Replicas:          2
    Updated Replicas:  2
  Conditions:
    Last Transition Time:  2023-12-28T17:12:00Z
    Reason:                Deploy
    Status:                True
    Type:                  ZookeeperReady
    Last Transition Time:  2023-12-28T17:12:20Z
    Reason:                Deploy
    Status:                True
    Type:                  BookKeeperReady
    Last Transition Time:  2023-12-28T17:11:56Z
    Reason:                Deploy
    Status:                True
    Type:                  PulsarBrokerReady
    Last Transition Time:  2023-12-28T17:12:20Z
    Reason:                AllConditionStatusTrue
    Status:                True
    Type:                  Ready
    Last Transition Time:  2023-12-28T17:10:05Z
    Reason:                Ready
    Status:                True
    Type:                  PulsarInstanceReady
  Zookeeper:
    Ready Replicas:    3
    Replicas:          3
    Updated Replicas:  3
Events:                    <none>

From the outputs, you can see that the status and type parameters for items under Conditions are set to true and ready respectively. This means that the neo-1 cluster is created successfully.

Configure the Service Context to interact with Pulsar and Kafka protocol

Starting from version v1.0.0, snctl introduces the concept of a Service Context. This feature aims to make snctl a unified tool not only for managing StreamNative Cloud resources (like creating and managing clusters) but also for directly interacting with the data plane of your Pulsar clusters using both the native Pulsar protocol and the Kafka protocol (via KSN or Ursa Engine).

A Service Context is essentially a named configuration profile stored within your snctl configuration (~/.snctl/config). It bundles the necessary connection details —- primarily the service URL and authentication information —- required to connect to a specific Pulsar cluster managed by StreamNative Cloud.

Automatic Context Discovery

Typically, you don't need to manually create contexts for your StreamNative Cloud Pulsar clusters. After successfully logging in using snctl auth login, snctl can often automatically detect the Pulsar clusters you have access to within your organization. It makes these clusters available as selectable contexts, usually using the instance name and cluster name as the context.

(Note: For managing contexts related to external, non-StreamNative Cloud clusters, see commands like snctl context add-external-context, snctl context list-external-context, etc.)

Switching and Using Contexts

To interact with a specific cluster using snctl pulsar or snctl kafka commands, you first need to activate its corresponding context.

  1. Set the Active Context: Use the snctl context use command, will likely prompt you to choose from a list of all availaible Pulsar Instances and Pulsar Clusters, select the target cluster by keyboard, and additional authentication may required if no valid token cached.

    snctl context use

    Non-interactive

    snctl context use --pulsar-instance $instance --pulsar-cluster $cluster

  2. Verify the Current Context: You can check which context is currently active using:

    snctl context current

    Output (Example)

    Using cloud service context
Pulsar instance: xxx
Pulsar cluster: xxx
Configured in organization: xxx
Current organization: xxx

Interacting with Pulsar and Kafka Protocols

Once a service context is active, all subsequent snctl pulsar client ..., snctl pulsar admin ..., snctl kafka client ..., and snctl kafka admin ... commands will automatically use the connection details (service URL, authentication) defined in that active context. You no longer need to specify connection flags repeatedly for these commands.

Running commands by using specific Service Account

While the active context usually derives its authentication from your user login (snctl auth login), you can override this for individual command executions to run as a specific service account. This is useful for automation or when specific permissions granted to a service account are needed. Another use case is to submit Pulsar Functions, Pulsar IO Connectors, and Kafka Connect connectors in StreamNative Cloud, this will makes the submitted instances running under given permission by the service account.

Crucially, snctl will verify if your currently logged-in user account has the necessary permissions to impersonate the chosen service account before executing the command.

  • Specify Service Account by Name: Use the --as-service-account flag followed by the service account name. snctl will use the credentials associated with this service account for the command.

    # Run a pulsar command as 'my-automation-sa' service account
snctl pulsar admin tenants list --as-service-account my-automation-sa

# Run a kafka command as 'my-kafka-processor-sa' service account
snctl kafka admin topics list --as-service-account my-kafka-processor-sa

  • Interactively Select Service Account: Use the --use-service-account flag without specifying a name. snctl will likely prompt you to choose from a list of available service accounts you have access to. This will shows you the details of each Service Account, as well as the status of each Service Account Binding, which is useful on manage Pulsar Functions, Pulsar IO Connectors, and Kafka Connect connectors.

    # Interactively select a service account for the command
snctl pulsar admin namespaces list --use-service-account

Example Workflow:

# Log in to snctl (if not already)
snctl auth login

# Switch to the desired cluster context
snctl context use my-production-cluster

# Verify the current context (optional)
# snctl context current-context

# Now interact with the cluster using Pulsar client commands
snctl pulsar client produce my-tenant/my-namespace/my-topic --message "Hello from snctl!"

# Or use Pulsar admin commands
snctl pulsar admin tenants list

# Or interact using Kafka client commands (if KSN is enabled, or if Ursa engine is enabled)
snctl kafka client consume my-kafka-topic --group my-group --from-beginning

# Or use Kafka admin commands
snctl kafka admin topics list

Note on pulsarctl Configuration:

This Service Context mechanism is internal to snctl for unifying its own commands. It is distinct from the snctl x update-pulsar-config command (described later). The update-pulsar-config command is specifically designed to configure the separate pulsarctl tool, allowing it to connect to your StreamNative Cloud cluster. The snctl context commands enable snctl itself to perform data plane operations directly via its pulsar and kafka subcommands.

Add clusters to the Pulsar configuration file

This section describes how to add a cluster to the Pulsar client configuration file. After adding a cluster to the Pulsar client configuration file, you can manage the cluster using the pulsarctl CLI tool. For details about how to manage Pulsar resources through pulsarctl, see pulsarctl command reference.

Note

It is recommended to use the pulsarctl context get command to set the context (cluster) in advance.

  1. Add a cluster to the Pulsar client configuration file.

    This example adds a neo-1 cluster to the Pulsar client configuration file.

    Input

    snctl x update-pulsar-config --cluster-name <neo-1>

    Output

    Updated Pulsar client configuration for context 'neo-1'.

  2. Verify that the current context has been changed.

    Input

    pulsarctl context current

    Output

    neo-1

After verifying that the change was made, you can use the pulsarctl CLI tool to interact with the target cluster. For details, see the pulsarctl reference docs. To work with tenants, namespaces, and topics, you can use the StreamNative Cloud Console. snctl does not currently support working with these features.

Previous
Tutorial
Next
Quick Reference