About StreamNative Operators for Apache Pulsar

StreamNative Operators for Apache Pulsar, or the Pulsar Operators, provide full lifecycle management of Pulsar deployments on Kubernetes. With the Pulsar Operators, you can easily deploy a Pulsar cluster on Kubernetes, create Pulsar-related resources, and manage them through the Kubernetes API and kubectl.

What are operators?

An operator is a custom controller in Kubernetes that uses Custom Resources (CRs) to manage different components of an application. It translates the specifications in a CR into concrete actions in Kubernetes. Specifically, it runs in a loop to monitor the resource state and detects if any change should be made. This ensures the current state matches the desired state for that specific resource.

To let Kubernetes identify CRs, you need to first define them using Custom Resource Definitions (CRDs), which extend Kubernetes capabilities beyond the default installation. A CRD manifest file contains all the configurations available for the CR that it defines. When creating the CR, you can choose to set the desired specifications so that the corresponding custom controller knows what configuration it should monitor.

By deploying a Kubernetes operator, you create the related CRDs and the custom controller. In Kubernetes, an operator works in the following way:

1. Apply the manifest file of a CR by using kubectl.
2. The operator (installed inside Kubernetes) deploys the resource.
3. The Kubernetes API server creates, updates, or deletes the corresponding containers and services based on the manifest file.

What are the Pulsar Operators?

The Pulsar Operators are custom controllers that provide a declarative API to simplify the deployment and management of Pulsar-related CRs on Kubernetes. The Pulsar Operators contain three individual operators that work together to manage components in a Pulsar cluster.

• Pulsar Operator: Manages the lifecycle of Pulsar brokers and Pulsar proxies. Broker Pods act as the servers in a Pulsar cluster to handle incoming requests from clients. Pulsar proxies, commonly used in private environments or Kubernetes, serve as the gateway to route external traffic to broker Pods.
• BookKeeper Operator: Manages the lifecycle of the BookKeeper cluster. BookKeeper nodes (bookies) store messages sent to the Pulsar cluster. You can configure storage for the ledger and the journal respectively in the BookKeeper CR manifest.
• ZooKeeper Operator: Manages the lifecycle of the ZooKeeper cluster. ZooKeeper stores critical metadata information about the Pulsar cluster and coordinates intracluster tasks.

By deploying the Pulsar Operators, you create custom controllers for the above three operators and CRDs for Pulsar brokers, Pulsar proxies, BookKeeper, and ZooKeeper.

Why should I use the Pulsar Operators?

The Pulsar Operators provide an application-specific way to automate complex tasks in Kubernetes. With the Pulsar Operators, you can easily carry out the following tasks on Kubernetes:

• Deploy and manage multiple Pulsar clusters on a single Kubernetes cluster.
• Configure a Pulsar cluster with the operator API.
• Perform rolling updates of a Pulsar cluster with zero downtime.
• Integrate a Pulsar cluster with the Function Mesh Operator.

How do I install the Pulsar Operators?

There are two ways to install the Pulsar Operators.

• Install the Pulsar Operators using the Helm chart. StreamNative develops the Helm chart to install the Pulsar Operators. As all three Operators are packed into one chart template, you can easily deploy them with one command and customize parameters like replicas as needed. This method requires you to manually upgrade CRDs when upgrading the Pulsar Operators, since Helm does not automatically upgrade CRDs when you use helm upgrade. For more information, see Upgrade the Pulsar Operators.
• Install the Pulsar Operators using the Operator Lifecycle Manager (OLM). The OLM provides a declarative way to install, manage, and upgrade Operators and their dependencies in a Kubernetes cluster. Compared with the deployment through the Helm chart, this approach does not require you to manually upgrade CRDs when upgrading Pulsar Operators.

After you install the Pulsar Operators, you can quickly spin up a Pulsar cluster by creating related CRs. For more information, see Deploy a Pulsar cluster using the Pulsar Operators.