Enable protocol handlers on Pulsar brokers

Apache Pulsar supports a pluggable protocol handler mechanism on its Pulsar brokers. By using protocol handlers, you can use Pulsar features with other messaging protocols like AMOP, Apache Kafka, and MQTT.

StreamNative brings the following protocol handlers:

Kafka-on-Pulsar (KoP) KoP is a protocol handler to natively support the Apache Kafka protocol on Pulsar brokers. By adding the KoP protocol handler to your Pulsar cluster, you can migrate your existing Kafka applications and services to Pulsar without modifying the code.
MQTT-on-Pulsar (MoP) MoP is a protocol handler to natively support the MQTT protocol on Pulsar brokers. By adding the MoP protocol handler to your Pulsar cluster, you can migrate your existing MQTT applications and services to Pulsar without modifying the code.
AMQP-on-Pulsar (AoP) AoP is a protocol handler to natively support the AMQP protocol on Pulsar brokers. By adding the AoP protocol handler to your Pulsar cluster, you can migrate your existing AMQP applications and services to Pulsar without modifying the code.

This document describes how to enable protocol handlers (AoP, KoP, and MoP) on Pulsar brokers running on Kubernetes.

Prerequisites

You have deployed a Pulsar cluster using the Pulsar Operators.

Enable protocol handlers

Note

If you have deployed broker Pods on your Kubernetes cluster, the manifest will overwrite the existing configuration and recreate the broker Pods. You can customize other fields in the manifest as required.
If you haven’t installed any components of Pulsar, you can create the pulsar namespace first and apply any of the following files to deploy brokers, BookKeeper, and ZooKeeper, with the related protocol handler enabled.

To enable protocol handlers, follow these steps:

Apply a PulsarBroker custom resource manifest with spec.config.protocolHandlers.<component>.enabled set to true.

cat <<EOF | kubectl apply -f -
apiVersion: pulsar.streamnative.io/v1alpha1
kind: PulsarBroker
metadata:
  name: brokers
  namespace: pulsar
spec:
  image: streamnative/sn-platform-slim:2.10.3.4
  pod:
    resources:
      requests:
        cpu: 200m
        memory: 512Mi
    securityContext:
      runAsNonRoot: true
  config:
    protocolHandlers:
      <component>:           # [1]
        enabled: true        # [2]
        proxyEnabled: true   # [3]
  replicas: 2
  zkServers: zookeepers-zk:2181
EOF

[1] component: represent the protocol handler (KoP, MoP, or AoP) to be enabled. Available options are kop, aop, and mop.
[2] enabled: enable the protocol handler (KoP, MoP, or AoP).
[3] proxyEnabled: access the MoP or AoP protocol handler outside a Kubernetes cluster. The Pulsar proxy will forward your requests to the Pulsar broker. This property is only available for MoP and AoP.

View the status of broker Pods.

kubectl get pods -n pulsar

You should see the following output:

NAME                         READY   STATUS    RESTARTS   AGE
bookies-bk-0                 1/1     Running   0          6m17s
bookies-bk-1                 1/1     Running   0          6m16s
bookies-bk-2                 1/1     Running   0          6m16s
bookies-bk-auto-recovery-0   1/1     Running   0          5m31s
brokers-broker-0             1/1     Running   0          87s
brokers-broker-1             1/1     Running   0          2m17s
proxys-proxy-0               1/1     Running   0          8m16s
proxys-proxy-1               1/1     Running   0          8m16s
zookeepers-zk-0              1/1     Running   0          8m21s
zookeepers-zk-1              1/1     Running   0          8m21s
zookeepers-zk-2              1/1     Running   0          8m21s