Deploy a Pulsar cluster using the Pulsar Operators

The Pulsar Operators provide full lifecycle management for all the components within a Pulsar cluster. You can use it to create, upgrade, and scale a cluster. This document demonstrates how to deploy a Pulsar cluster on Kubernetes using the Pulsar Operators.

A typical Pulsar cluster consists of the following components:

• Pulsar brokers: Handle and balance messages from producers, dispatch messages to consumers, and communicate with the Pulsar configuration store to handle various tasks.
• A BookKeeper cluster: Consists of one or more bookies for persistent storage of messages.
• A ZooKeeper cluster: Coordinates tasks between Pulsar clusters.
• (Optional) Pulsar proxies: Help route external requests to Pulsar brokers so that clients do not need to know the IP addresses of Pulsar brokers. Proxies are commonly used when direct interaction between clients and brokers is impossible. You can choose to deploy Pulsar with or without proxies.

Prerequisites

• You have installed the Pulsar Operators using the Helm chart or OLM.
• Configure a default Storage Class on your Kubernetes cluster. You can run the kubectl get sc command to view available Storage Classes.

Deploy a Pulsar cluster

By applying a single YAML file that contains the Custom Resources (CRs) of all required components, you can easily create a Pulsar cluster.

1. Create a Kubernetes namespace to deploy Pulsar Custom Resources (CRs). By default, Pulsar CRs are deployed in the pulsar namespace as specified in the YAML files in the next step. If you change the namespace in this step, you need to use the same namespace in the next step. This namespace can be different from the one where the Pulsar Operators are installed.

   kubectl create namespace pulsar
   

2. Install Pulsar.

   • Install Pulsar with Pulsar proxies.

     kubectl apply -f https://raw.githubusercontent.com/streamnative/charts/master/examples/pulsar-operators/proxy.yaml
     

   • Install Pulsar without Pulsar proxies.

     kubectl apply -f https://raw.githubusercontent.com/streamnative/charts/master/examples/pulsar-operators/quick-start.yaml
     

   Note

   Both YAML files contain the manifests for ZooKeeperCluster, BookKeeperCluster, and PulsarBroker. If you want to customize certain parameters like replicas, you can download the YAML file locally, make any adjustment as needed, and then apply it.

3. Verify that all components of the Pulsar cluster are up and running.

   kubectl get pods -n pulsar
   

   You should see the following output:

   NAME                         READY   STATUS    RESTARTS   AGE
   bookies-bk-0                 1/1     Running   0          6m20s
   bookies-bk-1                 1/1     Running   0          6m20s
   bookies-bk-2                 1/1     Running   0          6m20s
   bookies-bk-auto-recovery-0   1/1     Running   0          5m29s
   brokers-broker-0             1/1     Running   0          68s
   brokers-broker-1             1/1     Running   0          68s
   proxys-proxy-0               1/1     Running   0          8m25s
   proxys-proxy-1               1/1     Running   0          8m25s
   zookeepers-zk-0              1/1     Running   0          8m27s
   zookeepers-zk-1              1/1     Running   0          8m27s
   zookeepers-zk-2              1/1     Running   0          8m27s
   

After the Pulsar cluster is ready, you can use it to serve external requests.