Access Cloud Clusters using Kafka Clients with OAuth 2.0

Use the following information to configure your Kafka clients to use the OAuth2 authentication mechanism for connecting to StreamNative Cloud clusters.

Note

The OAuth2 authentication mechanism is currently only validated for Java clients with io.streamnative.pulsar.handlers.kop.security.oauth.OauthLoginCallbackHandler callback handler.

OIDC Federation is not fully supported for Kafka clients yet. The support is under development.

Prerequisites

Apache Kafka client: 3.2.1 or later

Include the following dependencies in your pom.xml file:

<dependency>
  <groupId>io.streamnative.pulsar.handlers</groupId>
  <artifactId>oauth-client</artifactId>
  <version>3.1.0.1</version>
</dependency>

Service URLs

In order to connect to a StreamNative Cloud cluster, you need to get its service URLs.

You can get a cluster's service URLs by following the steps below:

Navigate to the Cluster Workspace page.
Navigate to the Details tab, and in the Access Points section, you can find all the available service URLs of this cluster. Click Copy at the end of the row of the service URL to copy the URL.

Kafka Service URL (TCP): The URL of Kafka service.
Kafka Schema Registry URL (HTTPS): The URL of Kafka schema registry service.

JAAS configuration options

Before configuring your Kafka clients to use OAuth 2.0 for connecting to StreamNative Cloud clusters, you need to prepare a JAAS configuration for your clients by following the steps below.

Several configuration options are available for the callback handler. Sensitive configuration options and SASL extensions are included in the JAAS configuration file (sasl.jaas.config) while the others are top-level configurations.

JAAS Configuration Option	Description
`oauth.issuer.url`	The URL of the authentication provider which allows the Kafka client to obtain an access token. Currently, StreamNative Cloud only support Auth0 as the identity provider. So the value here should be `https://auth.streamnative.cloud`.
`oauth.audience`	The OAuth 2.0 resource server identifier for a Pulsar cluster. In StreamNative Cloud, a Pulsar cluster is identified by a Uniform Resource Name (URN), which is in the following format `urn:sn:pulsar:${your_orgnization_id}:${instance_name}`.
`oauth.credentials.url`	The URL to the JSON credentials file. It supports the following pattern formats: <br /> <li> `file:///path/to/file` </li><li> `data:application/json;base64,<base64-encoded value>` </li>

Configure Kafka Clients

The section describes how to configure Kafka clients to use OAuth 2.0 for connecting to StreamNative Cloud clusters. In this doc, we use urn:sn:pulsar:my_org:my_instance as the instance for an example.

For StreamNative OAuth2, audience is required.

import io.streamnative.pulsar.handlers.kop.security.oauth.OauthLoginCallbackHandler;

// replace these configs with your cluster
String serverUrl = "YOUR-KAFKA-SERVICE-URL";
String keyPath = "YOUR-KEY-FILE-ABSOLUTE-PATH";
String audience = "YOUR-AUDIENCE-STRING";

final Properties props = new Properties();
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, serverUrl);
...
props.setProperty("sasl.login.callback.handler.class", OauthLoginCallbackHandler.class.getName());
props.setProperty("security.protocol", "SASL_SSL");
props.setProperty("sasl.mechanism", "OAUTHBEARER");
final String jaasTemplate = "org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required"
    + " oauth.issuer.url=\"%s\""
    + " oauth.credentials.url=\"%s\""
    + " oauth.audience=\"%s\";";
props.setProperty("sasl.jaas.config", String.format(jaasTemplate,
    "https://auth.streamnative.cloud/",
    "file://" + keyPath,
    audience
));

Examples

See Connect to your cluster using the Kafka Java client for how to connect your Kafka clients to StreamNative Cloud clusters using OAuth 2.0.