1. How-to guides

Enable protocol handlers on Pulsar brokers

Apache Pulsar supports a pluggable protocol handler mechanism on its Pulsar brokers. By using protocol handlers, you can use Pulsar features with other messaging protocols like AMOP, Apache Kafka, and MQTT.

StreamNative brings the following protocol handlers:

  • Kafka-on-Pulsar (KoP) KoP is a protocol handler to natively support the Apache Kafka protocol on Pulsar brokers. By adding the KoP protocol handler to your Pulsar cluster, you can migrate your existing Kafka applications and services to Pulsar without modifying the code.
  • MQTT-on-Pulsar (MoP) MoP is a protocol handler to natively support the MQTT protocol on Pulsar brokers. By adding the MoP protocol handler to your Pulsar cluster, you can migrate your existing MQTT applications and services to Pulsar without modifying the code.
  • AMQP-on-Pulsar (AoP) AoP is a protocol handler to natively support the AMQP protocol on Pulsar brokers. By adding the AoP protocol handler to your Pulsar cluster, you can migrate your existing AMQP applications and services to Pulsar without modifying the code.

This document describes how to enable protocol handlers (AoP, KoP, and MoP) on Pulsar brokers running on Kubernetes.

Prerequisites

You have deployed a Pulsar cluster using the Pulsar Operators.

Enable protocol handlers

Note

To enable protocol handlers, follow these steps:

  1. Apply a PulsarBroker custom resource manifest with spec.config.protocolHandlers.<component>.enabled set to true.

    cat <<EOF | kubectl apply -f -
    apiVersion: pulsar.streamnative.io/v1alpha1
    kind: PulsarBroker
    metadata:
      name: brokers
      namespace: pulsar
    spec:
      image: streamnative/sn-platform-slim:2.10.3.4
      pod:
        resources:
          requests:
            cpu: 200m
            memory: 512Mi
        securityContext:
          runAsNonRoot: true
      config:
        protocolHandlers:
          <component>:           # [1]
            enabled: true        # [2]
            proxyEnabled: true   # [3]
      replicas: 2
      zkServers: zookeepers-zk:2181
    EOF
    
    • [1] component: represent the protocol handler (KoP, MoP, or AoP) to be enabled. Available options are kop, aop, and mop.
    • [2] enabled: enable the protocol handler (KoP, MoP, or AoP).
    • [3] proxyEnabled: access the MoP or AoP protocol handler outside a Kubernetes cluster. The Pulsar proxy will forward your requests to the Pulsar broker. This property is only available for MoP and AoP.
  2. View the status of broker Pods.

    kubectl get pods -n pulsar
    

    You should see the following output:

    NAME                         READY   STATUS    RESTARTS   AGE
    bookies-bk-0                 1/1     Running   0          6m17s
    bookies-bk-1                 1/1     Running   0          6m16s
    bookies-bk-2                 1/1     Running   0          6m16s
    bookies-bk-auto-recovery-0   1/1     Running   0          5m31s
    brokers-broker-0             1/1     Running   0          87s
    brokers-broker-1             1/1     Running   0          2m17s
    proxys-proxy-0               1/1     Running   0          8m16s
    proxys-proxy-1               1/1     Running   0          8m16s
    zookeepers-zk-0              1/1     Running   0          8m21s
    zookeepers-zk-1              1/1     Running   0          8m21s
    zookeepers-zk-2              1/1     Running   0          8m21s
    
Previous
Manually scale a Pulsar cluster