Manage Serverless Clusters on StreamNative Cloud

Create a cluster

Console
Terraform

You can follow the steps in Create a Serverless Instance to create a Serverless instance and its first cluster in it.When you create a resource in a Serverless instance, you can choose a Pulsar cluster or a Kafka cluster.

Serverless Kafka clusters are available only in supported serverless cloud providers and regions. If the selected serverless environment does not support Kafka clusters, the Cloud Console disables Kafka cluster creation and asks you to choose another environment or contact StreamNative support.

For Serverless Kafka clusters, StreamNative manages capacity automatically. The Cloud Console shows the current Serverless ETU capacity and limits during cluster creation. Self-service creation currently uses a 1 ETU base capacity, and the cluster scales automatically within Serverless limits. For capacity details, see Serverless ETU capacity and limits.

Creating a serverless cluster is straightforward. You just need to select a region and define the cluster in the Terraform configuration file.

resource "streamnative_pulsar_instance" "test-serverless" {
  ...
}

resource "streamnative_pulsar_cluster" "test-serverless" {
  depends_on      = [streamnative_pulsar_instance.test-serverless]
  organization    = streamnative_pulsar_instance.test-serverless.organization
  display_name    = "<your-cluster-name>"
  instance_name   = streamnative_pulsar_instance.test-serverless.name
  location        = "us-central1"
}

organization: The organization ID.
display_name: The display name of the cluster. Replace <your-cluster-name> with the actual name.
instance_name: The name of the instance where the cluster is created.
location: The location of the cluster. In this example, it is us-central1.

See PulsarCluster for more information.

View cluster details

Console
snctl
Terraform

Navigate to the Clusters Dashboard page.

On the Dashboard tab, you can view some metrics about the cluster, including the number of topics, subscriptions, consumers, producers, throughput, storage size, and backlog size.
On the Details tab, you can view the details about the cluster, including the cluster name, location, availability mode, cloud provider, cluster type, service URLs, the features enabled on the cluster, and your Pulsar, BookKeeper, and ZooKeeper versions.

To list all the clusters available for an organization, run the following command:

snctl get pulsarclusters -O <your-org-id>

If you want to get more details about a cluster, you can run the following command:

snctl get pulsarcluster <your-cluster-name> -O <your-org-id>

If you want to get more details about a cluster, you can define a data source in the Terraform configuration file.

data "streamnative_pulsar_cluster" "test-cluster" {
  organization = "<your-organization>"
  name = "<your-cluster-name>"
}

You can checkout PulsarCluster for more information.

Update a cluster

Serverless clusters are fully autonomous and automatically scaled. You don’t need to manually update them.

Monitor a Serverless Kafka cluster

For Serverless Kafka clusters, the Cloud Console dashboard shows Kafka metrics for the selected cluster, including topics, partitions, throughput, message rate, storage, connections, and consumer groups. StreamNative scopes these dashboard metrics to the selected Kafka cluster. This means that metrics shown on the cluster dashboard represent only that Kafka cluster, even when StreamNative manages the underlying infrastructure for you. For metrics API access and supported Kafka metric categories, see Cluster Metrics.

Delete a cluster

You cannot delete a cluster if there are resources associated with the cluster.

Console
snctl
Terraform

Navigate to the Clusters page.
Click the ellipsis at the top right corner of the cluster card that you want to delete, and then click Delete.
In the Are you sure you want to delete this? dialog, enter the cluster name and then click Confirm.

There are two ways to delete a cluster.

Delete the cluster by the cluster name.

snctl delete pulsarcluster <your-cluster-name>

Delete the cluster by the cluster manifest file cluster.yaml.
snctl delete -f cluster.yaml

Remove the cluster resource from the Terraform configuration file and run terraform apply to delete the cluster.

Get Started

Clusters

Data Streams

Process

Connect

Lakehouse

Governance

Pulsar Clients

MQTT Clients

Tools

Pulsar Changelogs

Manage Serverless Clusters on StreamNative Cloud

Create a cluster

View cluster details

Update a cluster

Monitor a Serverless Kafka cluster

Delete a cluster

​Create a cluster

​View cluster details

​Update a cluster

​Monitor a Serverless Kafka cluster

​Delete a cluster

Create a cluster

View cluster details

Update a cluster

Monitor a Serverless Kafka cluster

Delete a cluster