Configure KSN

Kafka on StreamNative(KSN) is built based on open-source KoP with a truly cloud-native experience, and enterprise-grade features to unleash developer productivity and operation efficiently:

• All of the Kafka features you're used to: KSN supports all the Kafka features you're accustomed to, including KStreams, KSQL, KTables with Topic Compaction, Schema Registry for the Java Client, and Kerberos Authentication for Kafka Clients.
• Great developer experience: KSN features streamlined local testing, with its own testcontainers module.
• Robust and reliable: Architecturally designed for scale and subjected to rigorous resilience testing. We've fortified KSN through comprehensive testing, ensuring it meets and exceeds the demands of large-scale Kafka operations.

Enable KSN

To use KSN on StreamNative Private Cloud, you need to configure on the PulsarBroker CR:

spec:
  config:
    protocolHandlers:
      kop:
        enabled: true
    transactionEnabled: true

• [1] spec.config.protocolHandlers.kop.enabled: Required. A bool to control the KSN enable or disable on Pulsar Broker.
• [2] spec.config.transactionEnabled: Required. A bool to control the transaction enable or disable on Pulsar Broker. KSN will need this to enbale the kafkaTransactionCoordinatorEnabled configuration.

Enable KSN Proxy

Kafka on StreamNative supports Pulsar Proxy extension which allows Kafka clients to connect and route Kafka traffic to Pulsar Broker with KSN enabled.

To use KSN Proxy on StreamNative Private Cloud, you need to configure on the PulsarProxy CR:

spec:
  config:
    kopProxy:
      enabled: true
      kafkaAdvertisedListener: 'a93bbde015ab54756ab4ce46154d7e8f-a6056a28f6b1b492.elb.us-west-2.amazonaws.com'

• [1] spec.config.kopProxy.enabled: Required. A bool to control the KSN Proxy enable or disable on Pulsar Proxy.
• [2] spec.config.kopProxy.kafkaAdvertisedListener: Required. The listeners for Kafka clients to use, and in Kubernetes environments you should configure the exteranl Proxy LoadBalancer type service address or NodePort type service address for this field.

Use Kafka Client to connect to KSN Proxy

After enabling the KSN on PulsarBroker and KSN Proxy on PulsarProxy, we can follow below steps to have a quick tutorial:

Download the Kafka release

tar -xzf kafka_2.13-3.6.0.tgz
cd kafka_2.13-3.6.0

Click Produce messages through the KSN Proxy

bin/kafka-console-producer.sh --topic Kafka-on-StreamNative --bootstrap-server a93bbde015ab54756ab4ce46154d7e8f-a6056a28f6b1b492.elb.us-west-2.amazonaws.com:9092

>First message to Kafka on StreamNative
>Second message to Kafka on StreamNative
>Third message to Kafka on StreamNative

Consume messages through the KSN Proxy

bin/kafka-console-consumer.sh --topic Kafka-on-StreamNative --from-beginning --bootstrap-server a93bbde015ab54756ab4ce46154d7e8f-a6056a28f6b1b492.elb.us-west-2.amazonaws.com:9092

First message to Kafka on StreamNative
Second message to Kafka on StreamNative
Third message to Kafka on StreamNative