Tutorials: Manage StreamNative Cloud resources using snctl
Work with organizations
An organization is a team account configured perfectly for your use case on the cloud provider of your choice.
Currently, snctl does not support creating organizations. You can create an organization through StreamNative Cloud Console. For details about how to create an organization, see create an organization.
After you have created an organization through StreamNative Cloud Console, you can use the snctl config set --organization
command to set your default organization.
With the organization
option, you need to use the random string for the organization name, not the descriptive name. You can find the random string on the Dashboard, next to the organization’s descriptive name. For an example, see organizations.
After setting the default organization, you can use snctl commands to perform other operations on StreamNative Cloud resources in the default organization without specifying the organization name.
Work with instances
In this section, we named the organization matrix
for an example.
Create an instance
A Pulsar instance is a group of clusters that acts together as a single unit.
To create an instance through snctl, follow these steps.
-
Define an instance named
neo
by using a manifest file and save the manifest fileinstance-neo.yaml
.-
This example shows how to create an instance on Google Cloud.
-
This example shows how to create an instance on AWS.
The following table lists fields in the manifest file.
Field Description apiVersion Specify the version of Pulsar API server. kind Specify the component to be created. -
-
Apply the manifest file to create the instance.
-
Check whether the instance is created successfully. This example shows whether the instance is successfully created on AWS.
Output
From the outputs, you can see that the
status
andtype
parameters for items underConditions
are set totrue
andready
. This means that the instanceneo
is created successfully.
In addition, you can use the snctl create pulsarinstance PULSAR_INSTANCE_NAME
command to create an instance. For details, see snctl reference.
Check the instance
You can use the following command to list all created instances.
Output
From the output of this command, you can see all created instances and the time when these instances were created.
Check instance details
Before checking details about an instance, you should use the following command to confirm whether the target instance is available.
Then, you can use the following command to check details about a specific instance.
The following example checks details about the instance neo
.
Output
Delete an instance
You can use the following command to delete a specific instance based on the instance name.
In addition, you can use the following command to delete a instance based on the type and name specified in the manifest file.
Work with clusters
A cluster is a secure messaging environment within Pulsar. Each Pulsar cluster consists a set of 3 components in a geographical location.
- Pulsar brokers - set of brokers handling all the data going in and out of Pulsar (or client requests)
- Metadata storage - providing coordination and service discovery between services
- Bookie ensemble - set of bookies that retain copies of the messages
A cluster has two layers: a stateless serving layer (made up of brokers) and a stateful storage layer (made up of bookies). See also Pulsar Architecture and Design.
In StreamNative Console, you can create one and only one cluster for an instance.
The code examples in this section use an organization called, matrix
.
Create a cluster
Before you can create a cluster through snctl
, you need to create a billing subscription. For more information, see subscribe to StreamNative Cloud.
To create a cluster, follow these steps.
-
Define a cluster named
neo-1
by using a manifest file and save the manifest fileclusterneo1.yaml
.The following table lists fields in the manifest file.
Field Description apiVersion Specify the version of Pulsar API server. kind Specify the component to be created.
-
Apply the manifest file to create the cluster.
-
Check whether the cluster is created successfully.
Output
Check the cluster
You can use the following command to list all created clusters.
Output
From the output of this command, you can see all created clusters and the time when these clusters were created.
Check cluster details
Before checking the details about a cluster, you should use the following command to confirm whether the target cluster is available.
Then, you can use the following command to check details about a specific cluster.
The following example checks details about the instance neo-1
.
Output
Interact with Clusters using Service Context
Starting from snctl
version v1.0.0
, the Service Context feature provides a streamlined way to interact directly with your Pulsar clusters (using both Pulsar and Kafka protocols) without needing to manually specify connection details for every command.
A Service Context stores the necessary service URL and authentication information for connecting to a specific cluster. snctl
typically automatically discovers your StreamNative Cloud Pulsar clusters after you log in (snctl auth login
) and makes them available as contexts.
1. Viewing and Switching Contexts
You usually don’t need to manually create contexts for your cloud clusters. To see which context is currently active and switch between available contexts:
-
Check the current active context:
Output (Example)
-
Switch to a different context (e.g., your cluster named
neo-1
):Output (Example)
Now, all subsequent
pulsar
andkafka
commands will target theneo-1
cluster.
(Note: For managing contexts related to external, non-StreamNative Cloud clusters, see commands like snctl context add-external-context
, snctl context list-external-context
, etc.)
2. Using pulsar
and kafka
Commands with the Active Context
Once a context is active, you can run pulsar
and kafka
commands directly:
-
Produce a message to a topic using the Pulsar protocol:
(Note: Replace
<organization_name>
with your actual organization name if not set as default) -
List Pulsar tenants using the admin API:
-
Consume messages from a topic using the Kafka protocol (requires KoP enabled on the cluster):
-
List Kafka topics using the admin API (requires KoP):
3. Running Commands as a Specific Service Account
Sometimes you need to perform actions as a specific service account rather than your logged-in user. snctl
allows this using flags, and it will verify your user has permission to impersonate the specified service account.
-
Run a command using a named service account (
--as-service-account
): -
Interactively choose a service account to run as (
--use-service-account
): This will prompt you to select from service accounts your user is allowed to impersonate.
Using Service Contexts simplifies interactions with your clusters directly through snctl
, making it a more powerful and unified tool.
Add clusters to the Pulsar configuration file
This section describes how to add a cluster to the Pulsar client configuration file by defining a context
for that cluster. Then, you can access the cluster using the pulsarctl
CLI tool.
-
Add a cluster to the Pulsar client configuration file.
This example adds a
neo-1
cluster to the Pulsar client configuration file.Input
Output
-
Verify that the current context has been changed.
Input
Output
Then, you can use the pulsarctl
CLI tool to interact with the target cluster. For details, see Connect to cluster through pulsarctl and perform other operations.
Delete a cluster
You cannot delete a cluster if there are resources associated with the cluster.
You can use the following command to delete a specific cluster based on the cluster name.
In addition, you can use the following command to delete the cluster based on the cluster name specified in the manifest file.
Work with service accounts
In this section, the organization has the name matrix
as an example name.
Create a service account
To create a service account through snctl, follow these steps.
-
Define a service account resource named
bot
by using a manifest file and save the manifest filesa-bot.yaml
.The following table lists fields in the manifest file.
Field Description apiVersion Specify the version of Pulsar API server. kind Specify the component to be created.
-
Apply the manifest file to create the service account.
Output
-
Check whether the service account was created successfully.
Output
From the output, you can see that the
status
andtype
parameters for items underConditions
are set totrue
andready
. This means that the service accountbot
was created successfully.
In addition, you can use the snctl create serviceaccount SERVICE_ACCOUNT_NAME
command to create a service account. For details, see snctl reference.
Download service account credentials
To use a service account, you first download its associated credentials to a JSON file. The information is made available through the status
block of the ServiceAccount
resource.
The following example shows how to download the service account credentials to a file called bot.json
.
The file contents will be similar to the following:
The following table lists two fields in the JSON file.
Field | Description |
---|---|
client_id | It is an Auth0 Application that has been created. |
client_secret | It is used to authenticate to Auth0 for accessing snctl. |
The file contains credentials information and should be well protected.
Download service account tokens
To get a token using the StreamNative CLI snctl
, run the following command.
Activate a service account
This example shows how to activate a service account through a key file called bot.json
.
Output
Access the StreamNative Cloud API
This example shows how to access the StreamNative Cloud API through a service account.
-
Log in to snctl.
-
Create a service account.
This example creates a service account named bot.
Output
-
Download the associated credentials of the service account to a JSON file。
Output
-
Bind the service account with an “admin” role.
Output
-
Log out from snctl.
-
Log in to snctl with the service account.
Output
Check service accounts
This example shows how to check the service accounts of an organization.
Output
From the output of this command, you can see all created service accounts and the time when these service accounts were created.
Check service account details
Before checking the details about a service account, you should use the following command to confirm whether the service account is available.
Then, you can use the following command to check details about a service account.
The following example checks the details about the service account bot
.
Output
Delete a service account
You can use the following command to delete a service account based on the service account name.
In addition, you can use the following command to delete the service account based on the name specified in the manifest file.
Work with API Keys
To work with API keys using snctl
, you need to snctl version 0.16.0 or later.
Create an API Key
To create an API key that doesn’t have an expiration date, do the following:
To create an API key that expires in 30 days, do the following:
To create an API key that expires in at a specific time, use the following:
View API Keys
To view API keys, use the following command:
Revoke an API Key
To revoke an API key, use the following:
Delete an API Key
To delete an API key, use the following:
Deleting an API key will also revoke it.
Work with users
In this document, we named the organization matrix
for an example
Create a user
To create a user, follow these steps.
-
Define a user named
ironman
by using a manifest file and save the manifest fileuser-ironman.yaml
.The following table lists fields in the manifest file.
Field Description apiVersion Specify the version of Pulsar API server. kind Specify the component to be created.
| spec | Specify the email address for the user. |
-
Apply the manifest file to create the user.
-
Check whether the user is created successfully.
Output
Check users
You can use the following command to check users created for an organization.
Output
Check user details
Before checking details about a user, you should use the following command to confirm whether the target user is available.
Then, you can use the following command to check details about a specific user.
The following example checks details about the user ironman@matrix.local
.
Output
Delete a user
You can use the following command to delete a specific user based on the user name.
In addition, you can use the following command to delete the cluster based on the type and name specified in the manifest file.