Connect to your cluster using the Kafka Stream
This QuickStart assumes that you have created a StreamNative cluster with the Kafka protocol enabled, created a service account, and granted the service account produce
and consume
permissions to a namespace for the target topic.
If you are using a Ursa-Engine powered cluster, please note that KStreams and KSQLDB support in Ursa Engine has certain limitations. It doesn’t support functionalities that require transactions and topic compaction.
This document describes how to connect to your StreamNative cluster using the Kafka Stream with OAuth2 authentication.
Before you begin
-
Get the OAuth2 credential file.
- On the left navigation pane, click Service Accounts.
- In the row of the service account you want to use, in the Key File column, click the Download icon to download the OAuth2 credential file to your local directory.
-
Get the service URL of your StreamNative cluster.
- On the left navigation pane, in the Admin area, click Pulsar Clusters.
- Select the Details tab, and in the Access Points area, click Copy at the end of the row of the Kafka Service URL (TCP).
- Create two topics for the kafka Stream application, named
<kafka-stream-application-name>-counts-store-repartition
and<kafka-stream-application-name>-counts-store-changelog
.
Steps
-
Add the Kafka Stream and OAuth Maven dependencies.
-
(Optional) Add the RocksDB dependency if you encounter the
java.lang.UnsatisfiedLinkError
error when starting the Kafka Stream application. -
Build a Kafka Stream application.
This example builds a Kafka Stream application named
wordcount-application
.serverUrl
: the Kafka service URL of your StreamNative cluster.keyPath
: the path to your downloaded OAuth2 credential file.audience
: theaudience
parameter is a combination of theurn:sn:pulsar
, your organization name, and your Pulsar instance name.
-
Run the Kafka Stream application to check the connectivity.
-
Open another terminal and use the Kafka CLI tool to send some messages to the
TextLinesTopic
topic. -
Type some texts.
Then, you should see the following input:
-