The Pulsar Operators provide full lifecycle management for all the components within a Pulsar cluster. You can use it to create, upgrade, and scale a cluster. This document demonstrates how to deploy a Pulsar cluster on Kubernetes using the Pulsar Operators.
A typical Pulsar cluster consists of the following components:
- Pulsar brokers: Handle and balance messages from producers, dispatch messages to consumers, and communicate with the Pulsar configuration store to handle various tasks.
- A BookKeeper cluster: Consists of one or more bookies for persistent storage of messages.
- A ZooKeeper cluster: Coordinates tasks between Pulsar clusters.
- (Optional) Pulsar proxies: Help route external requests to Pulsar brokers so that clients do not need to know the IP addresses of Pulsar brokers. Proxies are commonly used when direct interaction between clients and brokers is impossible. You can choose to deploy Pulsar with or without proxies.
- You have installed the Pulsar Operators using the Helm chart or OLM.
- Configure a default Storage Class on your Kubernetes cluster. You can run the
kubectl get sccommand to view available Storage Classes.
Deploy a Pulsar cluster
By applying a single YAML file that contains the Custom Resources (CRs) of all required components, you can easily create a Pulsar cluster.
Create a Kubernetes namespace to deploy Pulsar Custom Resources (CRs). By default, Pulsar CRs are deployed in the
pulsarnamespace as specified in the YAML files in the next step. If you change the namespace in this step, you need to use the same namespace in the next step. This namespace can be different from the one where the Pulsar Operators are installed.
kubectl create namespace pulsar
Install Pulsar with Pulsar proxies.
kubectl apply -f https://raw.githubusercontent.com/streamnative/charts/master/examples/pulsar-operators/proxy.yaml
Install Pulsar without Pulsar proxies.
kubectl apply -f https://raw.githubusercontent.com/streamnative/charts/master/examples/pulsar-operators/quick-start.yaml
Both YAML files contain the manifests for
PulsarBroker. If you want to customize certain parameters like replicas, you can download the YAML file locally, make any adjustment as needed, and then apply it.
Verify that all components of the Pulsar cluster are up and running.
kubectl get pods -n pulsar
You should see the following output:
NAME READY STATUS RESTARTS AGE bookies-bk-0 1/1 Running 0 6m20s bookies-bk-1 1/1 Running 0 6m20s bookies-bk-2 1/1 Running 0 6m20s bookies-bk-auto-recovery-0 1/1 Running 0 5m29s brokers-broker-0 1/1 Running 0 68s brokers-broker-1 1/1 Running 0 68s proxys-proxy-0 1/1 Running 0 8m25s proxys-proxy-1 1/1 Running 0 8m25s zookeepers-zk-0 1/1 Running 0 8m27s zookeepers-zk-1 1/1 Running 0 8m27s zookeepers-zk-2 1/1 Running 0 8m27s
After the Pulsar cluster is ready, you can use it to serve external requests.